Wayfinder Rating Incident

We have identified and are addressing an incident that has been affecting the Wayfarer rating system since April 2024. This issue caused some explorers’ ratings to gradually decrease over time, unrelated to their actual Wayfarer reviews.

We want to thank the community for bringing this to our attention both through the forum and in conversations at recent GO Fest events. Your feedback was crucial in helping us pinpoint the problem.

What We’ve Done:

  • Stopped the decline: We have implemented a fix to prevent the Wayfarer rating from incorrectly decreasing further.
  • Working on a full resolution: We are actively working to correct the underlying issue in the rating calculation.

Our Commitment:

We are committed to fully resolving this situation and restoring the Wayfarer rating system’s integrity. Our plan includes:

  1. Fixing the root cause: Correcting the error in the rating algorithm.
  2. Restoring ratings: Returning affected explorers’ ratings to their levels prior to the incident (if the prior rating was higher). We will not lower any ratings as part of this process.
  3. Compensating for lost agreements: Awarding the appropriate number of agreements to affected explorers to ensure their progress towards medals is not negatively impacted.

We sincerely apologize for the frustration and inconvenience this issue has caused. We deeply appreciate your continued feedback and support in helping us identify and address this problem.

We will provide updates as we make progress on the resolution. Thank you for your patience and understanding.

The Wayfarer Team

46 Likes

@NianticThibs I see where you say:

What feedback do you need to know if there is still an issue? What was causing the problem?

1 Like

OMG thank you for this info, I was going out of my mind trying to figure out what I was doing wrong. :joy: I even re-read the whole Criteria section, but… well, I know the criteria. :sweat_smile:
Good to hear this has been discovered and is being worked on!

6 Likes

Thank you for your update.

Following the investigation we now have the data and the reports needed to monitor the resolution of the incident. We already confirmed from those that rating decrease due to the incident has stopped. Thank you for your question.

As for the cause of the problem, we cannot share the details but it had to do with how we compute the rating when users agree/disagree with the submissions decisions.

14 Likes

I restarted the review the day after I got home from the Pokemon Go Fest in Sendai, and when I started, my ratings dropped rapidly. (Quickly drops from highest rank to fair) If this is true, I would like it back. Also, with the recent introduction of the ML learning function and the end of the wayfarer challenge, the backlog has been cleared, so the number of judges who review newly submitted wayspots, photo additions, titles, and descriptions is rapidly decreasing. I think. I thought it was because of this, but the results were not reflected well, and the agreements that produced the results were not reflected well. Please somehow return to the evaluation at the beginning of June. please.

2 Likes

When can we expect to have the rating fully restored and compensated?

5 Likes

To make this clear, what do you recommend us (affected reviewers) to do in the mean time?

Many of us stopped reviewing due to the uncertainty.

Can we proceed reviewing without fearing to loose anything (even with currently bad rating)?
Should we wait to review again until our rating is back in its original state?
Should we wait until further notice?

6 Likes

I am currently continuing the review regardless of the evaluation, but I felt that there might be some problem. If you feel there is a problem, I think it is better not to review.

2 Likes

Yesterday was a holiday in US (if I don’t remember wrong), now we are in the weekend and I guess the teams are going back from Madrid GO Fest so at least next week if everything works okay? If not I guess it will be later, but is not something we can know and I don’t expect them to say a closed date just in case they have problems resolving the problem.

2 Likes

Ah, so we really are being measured on how agreeable we are with everyone else as opposed to how thoroughly we actually reviewed things.

That’s a wonderful system you’ve created, truly great.

1 Like

Just found this announcement, but I am sad to find out my rating is still poor, so I still won’t dare to start reviewing again. It went from great to poor within 30 reviews, about an hour, and after I reached poor I did another 30 to see if it went up again, but alas.

The past days my number of agreements rose a bit, but my rating did not, and neither did my agreements for the Pokémon medal.

I have no idea if it is supposed to ‘fix itself’, or if I was wrongfully rated great and will now stay poor forever. Which kind of takes away the motivation to review again.

Do we have to continue reviewing to see improvement? Or will the algorithm be altered to get me back to my old level or at least back to good ?

2 Likes

The fix that the Niantic staff announced is not fully implemented yet, they said they will update with some progress soon in the initial post

5 Likes

I’ll keep my fingers crossed then. I assumed a correction of the algorithm would be more or less instant.

Thank you so much for this announcement! I have copied it to the people I know who were having problems! I’m just glad to know it’s all being fixed!

1 Like

Yeah, my rating finally went up back to good (after being great forever) and when I logged back in 30 minutes later without reviewing anything, it was already back down to fair.

3 Likes

Oh, I had the same - it went to good from fair so I thought YAY, did 6 reviews, back to fair :sweat_smile:

@NianticThibs

I miss a 4. on all those nominated wayspots that still remain without agreement bc voting players has bugged poor/fair rating.

None of the nominations I’ve submitted since April has gone through In voting by the community to agreements. So what’s happening with those? It’s not like wayfarers can vote on the nominations again, with the correct rating and impact.

Only AI or Niantic voted nominations has been resolved these two months, for my friends and I.

To be honest that is not my impression. It slowed down a little from less then a week during the competition to about ten days now (just got two more approved this morning I nominated on juni 13).

There’s probably a lot of differences depending on regions. I can only speak for northern Sweden and northern Finland. It’s slower than I have ever seen it.
Usually the In queue before voting have been longer than the In voting-period. Two nominations still In voting from mid April, more than two months ago, is unheard of since I started reviewing in 2018.
I used to have Great rating until after the wayfarer challenge. I’m on Poor now. Agreements moving slow, both for my nominations and for the nominations I’ve reviewed.

3 Likes