I’m new to reviewing and have two questions about description and location edits.
I’ve had a few description edits telling me which location is the right one and I don’t know if I should report them? The suggested description is clearly not a description of the waypoint but an instruction or information about which location I should choose. It seems like abuse of the functionality but the intention might not be abusive as they are trying to provide information about the edit?
Also, sometimes I see location edits with like 5 different suggested locations very close to each other, within a few meters or even less. I am afraid of picking any of them, even if one seems more correct than the others, because my rating will go down if I don’t pick the same one as the other reviewers? Why can there even be this many locations so close to each other? To me it seems like the edit doesn’t even make a difference and I feel like it’s a waste of my time dealing with these because I actually spend some time looking at street views and satellite views of these places to try and figure out which one is better.
This is abuse, so you can report them, or vote accordingly, which ever you prefer
For your point about multiple location changes - it could be that because our edits weren’t visible for years in contribution management that people have resubmitted things over multiple years. So not necessarily anything suspicious. It could also be that several people have submitted corrections and its never been voted on till now. I still have edits in voting from a year or 2 ago, and that’s not unusual
I would pick the best one, or say I don’t know for these. I don’t even know if we are sure we get agreements at all for edits
What you are describing in your first paragraph is indeed not the intended use of the description edit field. It has been clarified multiple times this should not be done and that action can be taken on such edits. The reasons why people put such edits in can be multiple and range from not knowing (there could still be old edits in the system from before it was made clear this should not be done and this could be someone trying to really fix a location) to abuse.
It is up to your discretion whether to report this (or anything in general, really) or not. I personally proceed on a case to case basis and try to establish whether the location edit is improving the existing location or not and whether the text seems to be used to mislead. Again taking the time for those investigations is completely voluntary and up to you. In any case these text edits should not be voted positively on.
Regarding rating drops, while it is true that there appears to be an unusual rating instability for some users at the moment (reported elsewhere on this forum), I don’t see why it would be specifically connected to location edits with multiple options. In large part this is due to the fact that until very recently edit reviews routinely took multiple years in some locations, and a lot of it is due to people thinking they just don’t do anything and repeating them over and over, or different users trying to put the same location correction in over multiple years. Some of these are abusive, some of these are not, overall they are just the product of a very backlogged system that is now functioning slightly differently. In the end my guess (I don’t know how this actually works) is that when you review these and when they finally resolve (again, can take years) you either align with the community, either not. If you don’t know and cannot tell which is correct there is a corresponding option you could pick
I don’t even know if we are sure we get agreements at all for edits
I’m not sure either, I just assumed so because my “Total Nominations Reviewed” goes up with edits aswell. But I don’t think they count in the agreements numbers because I am 5% into my second upgrade but I only have 51 total in my agreements out of 325 reviews.
It would be nice to see numbers for edits also because 51 out of 325 seems low because there are no numbers for the edits. It would be nice to include edit numbers or keep them out of the total reviews.
And regarding rating I haven’t had any issues so far, it is just a general concern because I want to keep my rating up.
Right now my rating is “Good” and I went down to “Fair” for a day or so and then I went back up.
But in general I wish there was a bit more transparency on how our rating is affected. For example, when I dropped to “Fair” I wondered if it was because I had done a lot of reviews that had not yet come to an agreement or if it had dropped because I didn’t agree with the community on nominations that had come to an agreement.
Will my rating drop if I do a lot of reviews in a short time and go back up as they come to agreements and I agree with the community?
@Xenopus I just came across an example of a description and title edit that has three valid titles worded differently and two valid locations.
The edit is on a playground and the three different titles have exactly the same meaning with a different order of the words and are all grammatically correct. The two locations are both clearly on the playground but in two different locations.
So with 6 different valid combinations of title and location there is a good chance I won’t agree with the community. Will this decrease my rating if I don’t choose the same as the majority?
These edits don’t seem to add or remove any value to the waypoint, but I can only choose to agree with one of the titles or disagree with all of them. And for the location I could choose one of them or say that I’m unable to find the appropriate location but I am able to say that both of them could be the appropriate location.
I wish I could skip this review because disagreeing with all of it is wrong and the edit itself is pointless but I have no option to avoid making a decision. And I don’t know how any of the options I have will affect my profile.
Yes. It would. That’s the unfortunate reality and a gigantic flaw in this whole system. You get punished for not having the same exact opinion / train of thought as everyone else.
I am pretty certain no one on this forum can tell you how the rating calculation works, and I also have no idea what your rating will do upon a certain action (and why wouldn’t it be a combination of actions instead, or a combination of actions at a certain time, or at a certain speed, or many other parameters I don’t have the imagination for). We don’t even know if it’s connected to agreements at all, to play devil’s advocate. I completely hear not liking the lack of transparency but I can also see very good and obvious reasons for us not to know how the rating works. So to answer your main question specifically on this edit scenario, I have no idea which choice nets you an agreement, but I would also like to understand your concern further. Are you specifically worried about disagreements on edit bundles because there are multiple combinatorial outcomes? Or about disagreeing with the community in general? If you are concerned about your rating drops of the last few days or weeks again then it’s way more likely related to an ongoing issue with ratings (linking to an acknowledgement by the Niantic team saying that it is under investigation) than to running into an edit bundle and agreeing on the title but disagreeing on the description. Or are you wondering in general if a single disagreement can drop your rating?
On that last option, since the only data we have to go on is “vibes” anyway, let me throw a few personal anecdotes in. I have received plenty of disagreements over the years on a daily basis, some of them likely for performing more investigations than the average reviewer, some of them likely from being more lenient on certain things than the communities I review in. I did not experience rating drops “from that” (though how would I know what a rating drop is exactly for, as I am not informed when things I review resolve and it could take years, but either way I did not experience them very much). Edit bundles have been a thing for a very long time, I have similarly reviewed a number of them.
It is hard to test this intentionally but over the years community members have occasionally been informed about things they reviewed resolving (chance things where it turns out to be the nomination of someone you interact with online and they post about their rejection or approval, for example), and from that it seems pretty clear that a single disagreement with whatever a nomination outcome was is not sufficient to move your rating.
@Xenopus Thank you for taking your time to reply, and sorry this is getting a bit off-topic from my original post.
I see the profile page we have as a performance based reward system consisting of three elements: the data showing us the number of agreements we made out of the total number of reviews, the performance rating and the reward being the upgrades we get. But, being acknowledged for your effort can also be a reward in itself so the rating could also be seen as a reward for some people.
Based on this the only assumption I can make is that my performance is evaluated based on the number of times I agree with the community rather than the quality of my reviews. The general rule is that you get what you measure, so lack of transparency might not be ideal as the measures leads to agreeing with any review rather than agreeing with the right review.
I understand how transparency can have adverse effects because it can be used for gaming the system, but lack of transparency can in my opinion have even bigger negative effects. If we are trying to do better as reviewers but still see our rating drop it leads to speculation and spending energy on frustration and trying to figure out how to review differently to get a better score rather than focusing our energy on doing better reviews. We might even start reviewing in an unwanted way because we see the majority of our local community reviewing this way because agreeing leads to a better score (again my assumption based on the data provided to me). So we start speculating what the community wants rather than following the guidelines when reviewing. I have also seen people making up their own rules for how and when to use the different likes or dislikes in specific scenarios even when that is not what the guideline says.
In a performance based reward system there has to be a clear connection between your performance and your rewards. And you have to be able to know how to increase your performance.
I really like the idea about wayfarer and I just want it to help people do well rather than keeping people from improving because some people can misuse the system (which I can see they are doing already). Transparency can also help identifying misuse just like we can report nominations and edits for abuse, maybe we should be able to report for review abuse. If we could actually see which nominations we got agreements on and which ones we got disagreements on, we would be able to report wrong agreements. But we would also be able to use it to improve our reviewing if we knew why we didn’t get the agreement.
Now to answer your question, I am not worried about disagreeing with the community in general. I want to review in the way that I see correct with regards to the guidelines. However, it is disincentivising to think that the right answer might decrease my rating even though I’m doing a good job. Also, thinking that my rating will decrease because some (and I see a lot) edits have multiple correct answers but only the one combination that the majority picked will actually increase my rating. And I have no way out of these edits because I can only disagree with all or agree with one and I can’t skip it.
For science I skipped through most nominations to get more of these edits. I tried to get out of these edits by disagreeing with all titles (because they were all correct) and picking the option that I’m not able to tell the correct location to get out of them in the most neutral way possible. And within minutes of doing this my rating dropped to fair again. So unless skipping also gives negative rating then I don’t see other ways to interpret these edits than “most likely lose situations”.
I think it’s a valid direction for the conversation to go in so no worries at all from a moderation standpoint.
Same, thanks for sharing your thoughts and thanks for the interesting discussion, I appreciate it. I will have more time to think about and reply to the deeper points later today, but for now I just want to point out that yes we are shown these items, and yes we are rewarded for agreements, but as it stands the fact that agreements are connected to the rating is one big assumption by the community and the following conclusions heavily hinge on that being true. This is not to be provocative (and I have zero insight into that on my end) and not to dismiss all the other concerns, this is just to highlight the fact that we literally don’t know.
Niantic said multiple times that rating is NOT based on agreements. Rating calculations look for whether you sped thru (for example maybe have a pattern to voting), or whether you seem to have considered each thing (for example, how you interacted with the page). Also whether you agreed with Niantic on nominations they pre-judged and threw into the system.
For OPR / Wayfarer’s first couple years, edits and photos were not included. (Only Niantic reviewed them.) We were told it was (at least partially) because with multiple choices it would be complicated to decide what is an agreement (used only for upgrades). Finally edits were introduced but not adding to agreements Niantic Wayfarer - with vague undocumented assurances that they’d look into edit agreements later. I never saw Niantic saying they’d changed this, or people observing that the agreement calculation had changed, and they were getting more than before.
On the Wayfarer Discussion Discord, people have reported many different agreement percentages at different ratings. Let me try to find some examples.
Hmm, it is hard to search on screenshots. But I see one posted where the user had a 68.6% agreement rate and a “Great” rating. And I slipped to “Good” for a few hours with a 78.2% agreement rate, which went back up to Great where it has stayed, but my agreement rate has slipped (a tiny bit) to 77.9%.
I think there is a correlation between agreements and rating, but I don’t think the agreements are the basis for rating.
Interesting, how do you know your agreement rate when edits are counted in your total?
I get a LOT of edits, my stats now are 45/10/7 agreements out of 413 reviews. So I have 15% agreement rate? I see this going up slowly all the time so idk if nominations are just moving slow here (in Denmark). But I feel like my number of agreements are very low considering the amount of reviews I’ve done. Again I don’t even know how many of these 413 are edits, I feel like it could be almost half of them?
My rating have been Good most of the time, never been in Great, and I guess I dropped to fair now because I’ve been trying out different strategies to get out of BS edits (see my posts above).
For reference, I started reviewing the last week of may and did my first nomination on 1. June. I have gotten 9 waypoints accepted by the AI and have quite a few “in voting” with one of them upgraded. Still waiting for my first to get accepted by voting but idk if this is considered slow, the oldest ones in voting are the ones I made on 1. June.
