Feedback for reviewers is needed

sogNinjaman-INGsogNinjaman-ING Posts: 3,295 ✭✭✭✭✭
edited March 2022 in April AMA - 2022

Currently, there is no feedback to reviewers as to how they are reviewing. We can look at some "home made" stats such as the percentage of reviews we have completed that agreed with the community vote, but that in no way links into our Wayfarer rating - you can have an excellent %age score while being "Poor". The whole Wayfare rating scoring system is very opaque and of no help to reviewers if we don't know what we are doing wrong.

I'd like to improve my "correct review" percentage, but in order to do so I need some sort feedback from Niantic. I understand that Niantic want to keep at least some of the criteria "flexible", but without any sort of feedback improvement is difficult. Can Niantic please provide some sort of feedback to reviewers so we can at least see say, the last 10 nominations we looked at where we did not "agree". Niantic don't have to actually specify what exactly is wrong, thus keeping their flexibility, but it might help us with a clue ( "See - I thought that generic Costa Coffee shop should have been rejected"). Given the large discussions about "mass rejections" and "bots", it might also let reviewers pick up on stuff like this quickly and report it to Niantic.

TLDR. Please can reviewers have some feedback on their "errors" from Niantic. YOu get a better Lightship database that way.

Post edited by NianticTintino-ING on
12
12 votes

New · Last Updated

Comments

  • Gazzas89-PGOGazzas89-PGO Posts: 2,031 ✭✭✭✭✭

    To add to this, maybe don't say "in the last 10 reviews, you didn't agree with 1", maybe instead show the pictures of the ones you did and say "one if these you didn't agree with rhe community" or even better, just show what wasn't agreed with and show why, for example you rejected something, but people said it was fine. Or you thought something was fine, but it was rejected for "bad photo" or "didn't meet criteria".


    The flip side thiugh is that you mught stop reviewing correctly and instead go with what the majority wants, but then that becomes the next topic in an ama

  • sogNinjaman-INGsogNinjaman-ING Posts: 3,295 ✭✭✭✭✭

    "The flip side thiugh is that you mught stop reviewing correctly and instead go with what the majority wants, but then that becomes the next topic in an ama."

    We are almost there with some things that the "local community" have decided should go through despite them not meeting Niantic criteria. Similarly, is it worth dmaging your stats and review level for voting "No" to things, for example, like the low quality, tatty, bare and broken "Noticeboard" - (Information sign / community hub / players meet up here daily) nomination that really should not make it into the game when 90% of reviewers are going to say "Info sign is an automatic 5* - you can't reject any of these". That is an issue with crowd sourced voting, if the "wrong" answer begins to be seen as "pointless to vote appropriately - go with the flow" you end up with stuff like modern EIIR UK Postboxes or Australian "Survey Markers" getting into the system.

  • RandomExploit-INGRandomExploit-ING Posts: 469 ✭✭✭✭

    Unfortunately agreements do not always equate to correct decisions. On many occasions the majority get a vote wrong or enough to cause a rejection and the percentage needed to reject is extremely low compared what's needed to accept in my opinion.

    The system relies on the wisdom of the crowd method of usually if you ask a large group the majority will know the right answer. This works better on facts though and not things which are open to interpretation or change like wayfarer submissions.

    So a simple example I will re-use for simplicity;

    A music or art school will get rejected because people see school and immediately go for a K12 rejection. 90% might do that for the example while 10% look properly and see it's an music school for the most gifted adult violinist's in the world from the description.

    Using agreements solely for feedback the 10% group which were right to accept it would be getting the email or feedback telling them they were wrong and should have rejected as K12 because 90% voted to reject and got the agreement's.


    So the system for feedback has to be done in a way so as to not re-inforce wrong decisions. The same could happen if you are in an area where groups or factions are voting to accept or reject for upgrades or team bias where the wrong decision gets fed back as the right one to others.


    I agree feedback needs to happen. But using agreements might be a flawed way of doing it. Such issues should be considered at least. Maybe it us not so much an issue once you consider the numbers needed for rejection Vs acceptance but only the Niantic god's know those

Sign In or Register to comment.