Why aren't appeals being used as a way to identify, evaluate, and educate poor reviewers?

AisforAndis-INGAisforAndis-ING Posts: 1,072 Ambassador
edited March 2022 in April AMA - 2022

One of the most desired aspects of appeals was that it would help curb bad reviewing and decrease the frequency at which nominations were improperly rejected by the community. Now that the appeals system is up and running, it appears that when Niantic processes an appeal, they essentially are only re-reviewing it. It does not appear that Niantic takes any look at who wrongly rejected a nomination, and it does not appear that Niantic is doing anything to identify those who are reviewing improperly and taking actions to educate those reviewers.

Why aren't appeals being used as a way to identify, evaluate and educate poor reviewers? Does Niantic have any plans to actually address bad reviewers in a way that will make a meaningful impact on the Wayfarer ecosystem?

Post edited by NianticTintino-ING on
82
82 votes

New · Last Updated

Comments

  • iFrankmans-INGiFrankmans-ING Posts: 268 Ambassador

    Yes similar to the question I posted about appeals: https://community.wayfarer.nianticlabs.com/discussion/30250/appeals-statistics#latest

    Statistics can provide valuable insights to improve the whole process.

  • RandomExploit-INGRandomExploit-ING Posts: 948 ✭✭✭✭✭

    Also, it needs to be done properly and take in to consideration people with a lot of agreements may be part of organised groups tactically voting, either for agreements or team bias.

    As well as prolific voters who have huge stats but also wrongly reject some things regularly

  • ElizionK-PGOElizionK-PGO Posts: 34 ✭✭✭

    Appeals are so new. We can’t expect Niantic to be using them as a way to deal with poor reviewers yet, but it would be nice to know if that’s a plan for the future. Perhaps we first need to ask What has Niantic learned from the initial round of Appeals? Are they seeing a high number of quality submissions that have been initially rejected by the community? If yes, what do they plan to do to right that ship? Is the next step to communicate with, redirect, penalize and/or re-educate those reviewers who are voting down submissions that then get approved on Appeal?

  • AgentMAenlight-INGAgentMAenlight-ING Posts: 227 Ambassador

    I agree with both of you. Penalties need to be done with care, and as AisforAndis-ING says, there are poor reviewers. I have had URLs in my supplemental information and rejected as "URL" too. I do not put the URL in my nomination's description. I have also been rejected for "mismatch location" for something that clearly exists. I have also had a university s rejected as "K-12". And many more. These inappropriate rejects frustrate the nominators.

  • AisforAndis-INGAisforAndis-ING Posts: 1,072 Ambassador

    I definitely understand the concerns about automatic punishements. If they go that route, and I do believe they should, it should be for things where the rejection was objectively incorrect, like an unwarranted URL, lisence plate, or face reject. And even then, it's probably best if an automated punishment only comes after multiple offenses.

  • DemonHunterCole-PGODemonHunterCole-PGO Posts: 42 ✭✭

    This needs to happen. Or make approved appeals viewable by all reviewers in a thread somewhere on these forums. Or both. It would help speedup the nomination process and also decrease the number of appeals filed.

  • Diskrepansi-INGDiskrepansi-ING Posts: 99 ✭✭✭

    It would also be so simple for some of the guidelines to be "built in" / included in the reviewing page. One quick example: the supplemental information section could include text saying "URLs are allowed in this section".

  • Diskrepansi-INGDiskrepansi-ING Posts: 99 ✭✭✭

    I think automation could happen after there are 'x' number of rejections over 'y' amount of time that end up successfully appealed.

    Example: in a 12-month period, 1-9 successfully appealed rejections notify the reviewer; 10th suspends reviewing for a month, or delays their own nominations from being approved, or suspends their ability to nominate, or whatever.

    That's just an example, obviously numbers would be modified to fit the overall situation based on much better analysis than I've done.

  • I strongly agree with @Diskrepansi-ING above. While I agree it's hard to see intention, setting a numeric bar of a certain number of mistakes in a certain time frame seems fair. A reviewer isn't making a genuine attempt to understand and get better if that type of pattern is being seen.

    I think the suspension is a good moderate first step but I would add it should also progress to a permanent ban if there are a certain number of suspensions in a time frame. (I.e. a year)

    Since the suspension(s) limit(s) the number of submissions an individual reviewer could do in the time frame, exceeding a set number of suspensions indicates at the very least that the reviewer isn't willing to try and understand what they're doing wrong and at worst reveals a deliberate attempt to act in bad faith, i.e. target agents they have a personal dislike of.

Sign In or Register to comment.