Why aren't appeals being used as a way to identify, evaluate, and educate poor reviewers?

AisforAndis-INGAisforAndis-ING Posts: 867 ✭✭✭✭✭
edited March 24 in April AMA - 2022

One of the most desired aspects of appeals was that it would help curb bad reviewing and decrease the frequency at which nominations were improperly rejected by the community. Now that the appeals system is up and running, it appears that when Niantic processes an appeal, they essentially are only re-reviewing it. It does not appear that Niantic takes any look at who wrongly rejected a nomination, and it does not appear that Niantic is doing anything to identify those who are reviewing improperly and taking actions to educate those reviewers.

Why aren't appeals being used as a way to identify, evaluate and educate poor reviewers? Does Niantic have any plans to actually address bad reviewers in a way that will make a meaningful impact on the Wayfarer ecosystem?

Post edited by NianticTintino on
80
80 votes

New · Last Updated

Comments

  • iFrankmans-INGiFrankmans-ING Posts: 140 ✭✭✭✭

    Yes similar to the question I posted about appeals: https://community.wayfarer.nianticlabs.com/discussion/30250/appeals-statistics#latest

    Statistics can provide valuable insights to improve the whole process.

  • RandomExploit-INGRandomExploit-ING Posts: 437 ✭✭✭✭

    Also, it needs to be done properly and take in to consideration people with a lot of agreements may be part of organised groups tactically voting, either for agreements or team bias.

    As well as prolific voters who have huge stats but also wrongly reject some things regularly

  • ElizionK-PGOElizionK-PGO Posts: 15 ✭✭✭

    Appeals are so new. We can’t expect Niantic to be using them as a way to deal with poor reviewers yet, but it would be nice to know if that’s a plan for the future. Perhaps we first need to ask What has Niantic learned from the initial round of Appeals? Are they seeing a high number of quality submissions that have been initially rejected by the community? If yes, what do they plan to do to right that ship? Is the next step to communicate with, redirect, penalize and/or re-educate those reviewers who are voting down submissions that then get approved on Appeal?

  • AisforAndis-INGAisforAndis-ING Posts: 867 ✭✭✭✭✭

    I completely agree that automatic punishments would have issues, however there are a lot of cases where people just blatantly did not review correctly. The frequency of this I'm sure varies greatly between different rejection reasons.

    For example, I recently had a nomination rejected for "URL" when the only URL in the submission was in my supplemental info. The people that rejected me for URL blatantly and objectively did not review correctly. Those people should be reeducated at the minimum.

    Some other examples include a submission in community garden that was rejected for PRP, a trail marker that was rejected for pedestrian access, a nature sign that was rejected for natural feature, etc. These type of things are in absolutely no way subjective, and when people are found to be repeatedly reviewing incorrectly, they should be punished.

    There are times where I can understand why a reviewer might reject one of my submissions for location mismatch (or give it a low location score), even when I know they are wrong. Every reviewer is different and some will be more trusting and may put in more effort than others. But sometimes rogue reviewers will swoop in with unjustified rejections and tip the scales against you with a bad rejection reason when they shouldn't have. Combatting those reviewers is what is most important, its objective, and in many cases it could make all the difference in whether a nomination gets approved or not.

  • AgentMAenlight-INGAgentMAenlight-ING Posts: 183 Ambassador

    I agree with both of you. Penalties need to be done with care, and as AisforAndis-ING says, there are poor reviewers. I have had URLs in my supplemental information and rejected as "URL" too. I do not put the URL in my nomination's description. I have also been rejected for "mismatch location" for something that clearly exists. I have also had a university s rejected as "K-12". And many more. These inappropriate rejects frustrate the nominators.

  • AisforAndis-INGAisforAndis-ING Posts: 867 ✭✭✭✭✭

    I definitely understand the concerns about automatic punishements. If they go that route, and I do believe they should, it should be for things where the rejection was objectively incorrect, like an unwarranted URL, lisence plate, or face reject. And even then, it's probably best if an automated punishment only comes after multiple offenses.

  • CipherBlakk-PGOCipherBlakk-PGO Posts: 308 ✭✭✭✭

    I don't think there's any way to properly automate punishment. At some point, an actual thinking person is going to have to look at and evaluate a reviewer's "bad" decisions and determine what the bad pattern actually is, and then approve punishment. Even then, the bad decisions would have to be so obviously ill-intentioned that there could be no disputing that, and that's incredibly hard to determine. Someone rejecting URLs, for example, might just have an improper understanding of the rules. And, since there's no situation where any given type of POI deserves an automatic approval, you can't have automatic punishments, either.

    I like the concept of showing someone a successfully appealed submission that was rejected by them. That contributes to the continuous learning. Automatic punishments (which will inevitably not be done properly) will just give Wayfarer a bad image. No one will want to bother with a punishment-happy platform. Remember, this is freely given labor Niantic is exploiting. They'd be looking a gift horse in the mouth if they automated punishment.

  • DemonHunterCole-PGODemonHunterCole-PGO Posts: 12 ✭✭

    This needs to happen. Or make approved appeals viewable by all reviewers in a thread somewhere on these forums. Or both. It would help speedup the nomination process and also decrease the number of appeals filed.

Sign In or Register to comment.