Improving the system: Appeal ONE rejection reason

WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭

General ideas described in https://community.wayfarer.nianticlabs.com/discussion/15568/improving-the-system

Appeal ONE rejection reason.

We've seen so many nominations rejected with totally wrong rejection reasons. This proposal doesn't suggest that Niantic would review and approve the nomination, only that you can appeal one of the rejection reasons in your nomination using for example one upgrade (it must cost something or otherwise everybody would appeal every nomination), then a Niantic's employee would review that rejection reason: a picture of a house rejected due to "live animal", the person that used that rejection reason is doing a bad job in Wayfarer, maybe the nomination is not valid due to other reasons, but that reason is bogus and it creates issues as people are unable to understand why their nomination has been rejected.

If Niantic agrees that the rejection reason is wrong then the reviewers that selected it are notified that they have failed a review, maybe hit them with a cooldown and force them to take the test again. If someone gets another strike for a bogus rejection again, then the cooldown would increase. The original nomination remains rejected.

Goal: prevent bogus rejection reasons, educate reviewers about criteria and in the end help the system by removing bad actors that force valid nominations to appear several times up for review.

Improving the system: Appeal ONE rejection reason 43 votes

Yes, One Upgrade = One appeal for One rejection reason
27%
Stultusz-INGTurckoPT-INGDerWelfe2205-PGOWheelTrekker-INGB00JL5YI7G-PGOPokegalAneesa-PGOfreebirdnerd-PGOeduKings95-PGOSamKwoko-PGOC0LERBEAR120-PGOVitoriosoGo-PGOTeamRocky15-PGO 12 votes
Yes. More than One Upgrade = One appeal for One Rejection reason
6%
SilverPT-PGOWoodWose-PGOtoniukupaoni-ING 3 votes
Yes, One or More upgrades = Appeal for Several rejection reasons
30%
LexTolkien-PGOGazzas89-PGOHaramDingo-INGDam0clees-PGOfpopp21-PGODaleTheWolfdog-PGOIMRAN592-PGOTWVer-INGpkmnsearch2-PGOAeryle88-PGORoepaf-PGOZireka-INGPhilPrado-PGO 13 votes
No
34%
Rodensteiner-PGOGearGlider-INGsogNinjaman-INGTheFarix-PGOTheisman-INGoscarc1-INGHosette-INGflatmatt-PGOWandHerring-PGO4Pro4IV-PGO52cucumbers-INGholdthebeer-INGKwyjiboHan-PGOLechu1730-PGODavid75Chu-PGO 15 votes

Comments

  • flatmatt-PGOflatmatt-PGO Posts: 2,137 ✭✭✭✭✭
    edited April 2021
    No

    I mean, I appreciate the aim, but I really can't imagine many people would be willing to spend an upgrade on this instead of upgrading their submission. The only people I could see using this are those who want to get rid of their upgrades just to avoid upgrading tough submissions, and there's a much better way to solve that problem (which would be not auto-applying upgrades). *Maybe* someone in an incredibly fast review area.

    I can also see it causing a lot of confusion along the lines of "Niantic accepted my appeal, where's my [Pokéstop/Portal]?!" given how inclined people are to read the details of anything.

  • holdthebeer-INGholdthebeer-ING Posts: 199 ✭✭✭✭
    edited April 2021
    No

    Honestly I don't expect Niantic to assign employees to deal with such appeals.

    Also, some rejection reasons are already controversial. You might want to punish reviewers for selecting "picture from car", but they selected "low quality photo" for other reasons, and it's combined in the email. Or, the infamous "not historical" rejection for trail markers. I'd like to appeal such rejections as bogus, trail markers are not expected to be historical. But can't imagine how reviewers should be punished for sharing the opinion.

    The improvement I want: show how many reviewers picked each rejection reason, percentage or rough estimate. Like, if nomination comes back rejected with:

    • Face on photo, 50%
    • Visually Unique, 5%
    • Culturally or historically insignificant, 10%
    • K-12 or school, 5%

    See? No need be obsessed with "bogus reasons", it's a fluke. Just need to take another picture.

    Or, if nomination is rejected with 90% "does not meet criteria", it does not matter what other reasons are listed. Need to focus on what criteria it meets, or submit something else.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    It seems that most of the people so far are against this idea.

    The problem for many people is that they get rejection reasons that are wrong and if they nominate the same candidate again, it might be reviewed again by the same people doing a bad job and it enters an endless loop of anger because they are unable to prevent those bogus reasons.

    By setting a "price" of one upgrade, it means that the nominator believes strongly that there's a person doing a bad job and he's willing to bet an upgrade to try to change that in the system.

    If they prefer to nominate again ignoring those rejection reasons then the problem might go on and the system won't improve at all.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    Right, not all the rejection reasons might lead to the possibility of an appeal, but in your case: don't you think that the person that selected K-12 for something that is not related to a school should get a warning?

    How can we improve the system if I'm allowed to reject anything selecting any rejection reason and nothing happens to me?

  • holdthebeer-INGholdthebeer-ING Posts: 199 ✭✭✭✭
    No

    "but in your case: don't you think that the person that selected K-12 for something that is not related to a school should get a warning?" - not really, would not waste my time or upgrades chasing that.

    I expect the current system will send them to poor rating or into cooldown if they do it consistently. I mean, there is room for error, misclick, different opinion, etc., - punishment should happen only if reviewer is bad enough - and this is where "reviewing patterns" detection and honeypots should work. Without data, there is no evidence how fast the current system reacts, maybe it's good enough.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    Yes, Niantic doesn't seem to be willing to invest in Wayfarer, otherwise the situation would be quite different.

    How many people are annoyed by incorrect rejection reasons? How many of them give up a system that doesn't work?

    Even if someone doesn't want to use this system, Why shouldn't there be an option for those that are sure that one rejection is totally wrong to appeal that? As it would require at least an Upgrade and it won't get the nomination approved most of the people won't use it anyway.

    Do you wanna know what bothers me?: those threads where people keep arguing back and forth if a rejection is legitimate or not. Allow the people to put their money where their mouth is. Use a hard earned upgrade to get a confirmation that the nomination shouldn't have been rejected due to that reason, and as that person had to use an upgrade, the people that used that rejection reason then should get a hit, maybe first time only a warning, and next time a cooldown, etc... the details don't matter at the moment, the idea is to allow a way to request a check on bad reviewers.

  • Hosette-INGHosette-ING Posts: 3,469 ✭✭✭✭✭
    No

    A rejection is a rejection and the reasons are less important than the fact that the submission was rejected by reviewer consensus.

    You do realize that a "wrong" rejection reason could be as simple as a misclick, right?

  • sogNinjaman-INGsogNinjaman-ING Posts: 3,313 ✭✭✭✭✭
    No

    A lot of people a) seem to take rejections very personally (unless Google translate is translating a but too "strongly" from some languages like Spanish into English and b) a lot of people don't actually seem to notice the "does not agree with criteria" message.

    I forsee a lot of people appealing the "poor photo" or "sensitive location" when in fact the real reason for the rejection is "does not meet criteria". We already have a section in the Forum for "Nomination Improvement" which essentially does the same thing. Niantic do not have the staff to deal with everybody under the sun appealing rejected nominations. As the standard message says "resubmit if you think it good enough", but get it commented in here first.

  • Lechu1730-PGOLechu1730-PGO Posts: 537 ✭✭✭✭
    No

    No, Google Translate isn't translating a bit too strongly, people do take personally. Nevertheless if you keep hanging around here you'll end up submitting much stronger nominators. I normally post my nominations (the local ones at least) to my raid group to avoid dupes. They are amazed at all the info I routinely add to nominations.

    More than appeals, I would like to see an approximate break down of rejection reason percentages. If Niantic don't want to use numbers they could use some indeterminate quantifiers (most, many, some, a few) associated to different thresholds (more than 60%, 20% - 60%, 6% - 20% and under 6%. That will let people focus on the important reasons.

  • HaramDingo-INGHaramDingo-ING Posts: 1,725 ✭✭✭✭✭
    Yes, One or More upgrades = Appeal for Several rejection reasons

    Let's assume that system has a 50% approval rate for a nomination to go through, and that every reviewer's votes are counted equally. If, out of 30 reviewers:

    • 12 people vote favourably,
    • 10 people vote against with Does Not Meet Criteria, but
    • 5 people vote against with Live Animal,
    • 2 people vote against with Explicit Content, and
    • 1 other person vote against with Abuse...

    Then because 60% of reviewers have voted against the nomination in general, then it will cause the nomination to be rejected.

    This appeal system might be able to turn the tide and cause a nomination to actually appeal its rejection because of these grossly inappropriate reasons if there is proof and irrefutable evidence that the nomination does not promote séxual services or abusive in any way, nor does the nomination's photo actually does focus on a live animal because of these troll reviewers. An appeal like this will nullify all the incorrect voting reasons, and because we see that more people have voted favourably, then hooray, the nomination is approved.

    On the other side however, appealing particular reasons such as natural feature, live animal and etc which would only hold a minority vote should a nomination have an against landslide against those who voted favourably (i.e. only 4 people voted favourably, but 22 people correctly voted with Does Not Meet Criteria, two people with Live Animal, one with Natural Feature and one for K-12 School). So appealing something like this would not make a difference anyway.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    If a person rejects a nomination with a wrong reason and that nomination ends up being rejected, then as far as we know that counts as an agreement for that person and they won't go into bad status.

    The person that sent the nomination gets back a non-sensical rejection reason and doesn't know how to improve their nomination.

    It's OK that you don't want to use your time or upgrades in this way, if this option would exist doesn't mean that anyone had to use it. Most of the people would accept the rejection, or send again the nomination, shout at the wall, whatever. Other people instead would prefer to try to achieve a change in the people that it's doing a bad job and when they are 100% sure that their nomination is valid and has been incorrectly rejected this would allow to appeal the rejection reasons one by one using their upgrades.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    That would be fine if the consensus was real and most of the people really rejected the nomination.

    The issue is that we have strong reasons to believe that a group quite smaller than the 50% can manage to reject a valid nomination, so maybe this could be somehow mixed with the suggestions by @Lechu1730-PGO and if the nomination has been rejected by more than 50% of the people then there's no possible appeal and allow appeals if the nomination did met a minimum threshold of positive votes.

    Yes, a misclick can lead to a wrong reason being selected. As long as you're not misclicking continuously then this shouldn't be a big issue as very few people would be willing to use one upgrade just to appeal one rejection reason and that the action wouldn't lead to the approval of their nomination. Only those that are quite sure that they sent a valid nomination and are annoyed by the repeated rejections would prefer to spend an upgrade this way instead of using it in a new nomination.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    Yes, using the community would be a great idea.

    Surely there are enough reviewers in good status that might be willing to take part in this kind of meta-review. After a person appeals a rejection, that nomination is sent to a group of reviewers taking part in the program, let's say a minimum of 5 persons, maybe even 7 or 10.

    They see the nomination with all the information mostly as usual, but this time instead of rating it they see the rejection reasons and they must assert if those reasons are correct or not. Any action would require a high agreement rate between this meta-group, and it would scale up to Niantic only when there's no agreement. This group could write also a message to the nominator / reviewers explaining why there's something wrong or what can be improved.

  • Aeryle88-PGOAeryle88-PGO Posts: 440 ✭✭✭
    Yes, One or More upgrades = Appeal for Several rejection reasons

    The "does not meet criteria" is too vague.

    And generally when we submit something, we submit a place that's meet the criteria.

  • sogNinjaman-INGsogNinjaman-ING Posts: 3,313 ✭✭✭✭✭
    No

    Therein lies the problem. Everybody thinks their nomination meets the criteria. Looking at my review stats, this is clearly not the case, around 50% of the submissions I get are rejected because they do not meet the criteria.

  • David75Chu-PGODavid75Chu-PGO Posts: 23 ✭✭
    No

    I don't think we should start punishing reviewers. In general we are putting in a lot of time for free to try and help improve the game. Despite the guidelines there is always room for some subjective interpretation, for example a professional photographer may have a stricter view than a casual selfie snapper on what a good or poor quality photo is.

    I think most of us are trying to do this properly.

    I have been surprised to read in the forums how angry some people seem to get due to rejections. Personally I would like to see so many approvals, but there have been some shockers ranging from random trees, to pylons, to bits of paper stuck on a fence, to photos that are so blurred that they make your eyes water. We don't want to introduce a system where we approve any old chaff just so we avoid warnings or punishments as the final quality would plummet.

    There are easy approves, and easy rejects, but as always there are trickier borderline cases that could go either way depending on different perspectives. It is going to always be difficult to 'police' such cases when relying on community spirit to make it work.

  • Hosette-INGHosette-ING Posts: 3,469 ✭✭✭✭✭
    No

    @Aeryle88-PGO writes:

    And generally when we submit something, we submit a place that's meet the criteria.

    That may be true for you but I can assure you that it is far from universal. Over 17K reviews I have accepted fewer than 50%. The rest either failed for not meeting basic acceptance criteria, or for hitting one of the rejection reasons (e.g. K-12 school, no safe pedestrian access.) I don't know how to explain the mismatch other than to think that a lot of people are submitting things that they think meet the criteria but really don't.

  • GearGlider-INGGearGlider-ING Posts: 1,335 ✭✭✭✭✭
    No

    For everyone person appealing a wrongfully rejected nominations, you’d have dozens appealing rightfully rejected ones. It would be a similar situation like when Niantic reviewed portals themselves a few years back, except you’d have to deal with more people complaining about how Niantic reviewers are the *insert insult* for rejecting their nomination.

  • Aeryle88-PGOAeryle88-PGO Posts: 440 ✭✭✭
    Yes, One or More upgrades = Appeal for Several rejection reasons

    Maybe.

    But for me, every time i get a rejection, i feel scammed by the system (not sure if we said that in english) because i check amny times all what i submit in order to avoid rejection.

    Despite that, i get rejection with false arguments (like living animal for a famous place with a sign) or with the vague "not meet criteria" despite the fact i'm sure my submissions meet the criteria.

    Every time i get a rejection, it's a lost submission and i had to wait 15 more days to submit again, and wait again for the vote, etc... That's very upsetting and frustrating.

    Allowing appels for rejections would be usefull to solve this problem.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    My proposal is not aimed at being an automatic fix for any rejected submission. Looking at your nomination of the equestrian center, we can see that no one has stated "yes, this is clearly a valid nomination", instead there's a lack of information, no way to validate that your statements in the nomination are correct.

    You know far more info about that location than the reviewers, and you should focus on how to send that information to the reviewers so no matter what's their previous knowledge about farms, equestrian centers and your town, they are able to agree with you.

    And there's a very important part in my proposal: even if you appeal a rejection, the goal is not to get that nomination approved, instead the goal is to find out if there are wayfarers that are doing a bad job selecting the rejection reasons, so in your case you might try to appeal the "doesn't have pedestrian access" as it obviously have pedestrian access, but the rejection because people think that it's a farm is due to the lack of information that you have provided to validate that it's an open center to the public.

  • Aeryle88-PGOAeryle88-PGO Posts: 440 ✭✭✭
    Yes, One or More upgrades = Appeal for Several rejection reasons

    Once again, about this nomination in particular: it's impossible to proove everything. Many submissions are validated without having to provide hundred of proof. (and curiously: the playground for children in the equestrian center were validated two day after the rejection of the carrer of the equestrian center, and no proof were needed to proove it's not a farm...)

    And if we were able to make an appel, we won't have to loose a new submission to give additionnal information that reviewers wants. In the case of the center: with an appel i can easely give information that's not a farm without having to wait 15 day to have submission again, go to the center again, take the same photo, write the same name and description, and just add a link that proove it's an equestrian center (despite it's obvious on the photo), and loose that submission for an other potential POI.


    With only 7 submission per 14 days, having to use one of them everytime something is rejected is very frustrating, especially when you have a list of more than 100 potential POI to submit. Make an appeal could be a solution.

  • sogNinjaman-INGsogNinjaman-ING Posts: 3,313 ✭✭✭✭✭
    No

    If the goal of an appeal is not to get the nomination approved, as 99% of the people using the facility would probably expect, then what is the point of being able to "appeal a rejection reason".

    If the point is try to evaluate the responses from individual reviewers and find those giving "wrong" reasons, even if the overall outcome for the submission is unchanged (ie I say "Animals" when the proper reason is "Obstructs emergency services") then your suggestion is not the way to go about it. Niantic already collate the rejection reasons and can see what each reviewer said, so if a reviewer was consistently selecting "wrong" reasons - those that do not agree with the majority rejection reasons, it would simple to pick these out. Then what? Do you provide feedback to both the submitter and reviewer, or do you "punish" the reviewer, as many seem to request. A very large number of "submission review" requests seem to be focussed on that - "My rejection is perfect, so anybody who rejected it for any reason should be punished".

    Whatever way you go about things, without some sort of feedback to reviewers from Niantic so they can improve before "punishment", there is no way of doing things fairly AND meeting the expectations of submitters, all of whom expect their nominations to pass, even if they themselves do not understand the criteria. The first stage in any of this would be some reviewer feedback from Niantic, which they seem very reluctant to provide or even consider.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    This comment confirms what has been stated by others: the expectations of people using this option wouldn't match its outcome, so it wont' work.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,387 ✭✭✭✭✭
    Yes, One Upgrade = One appeal for One rejection reason

    My latest thoughts taking into account the provided feedback changed the person from Niantic to a number of Wayfarers that opt-in into a "meta-wayfarers" group, and these persons would provide comments back to the person that sent the nomination and in case that a nomination had wrong rejections then those reviewers would get also feedback so they can adjust their behavior.

    Nevertheless, this proposal would need huge adjustments in order to be accepted by the community, so thank you everyone for your feedback and let's close it.

  • Jtronmoore-PGOJtronmoore-PGO Posts: 1,581 ✭✭✭✭✭

    You can’t have the mindset that you are 100% correct all the time you have to be willing to adjust your nominations some times to help them. I’ve seen you on multiple posts now and you complain everytime about the rejection reasons. Some of them are warranted sure but others make perfect sense because remember when you nominate something it goes to audiences that can be no were close to you (3+hour drive away). They don’t know the area. So what is “obvious” to you is not obvious to a reviewer seeing something the first time. Take a step back and don’t come into every thread with the mentality that your nominations are 100% the best, because they are not 🤦🏻

  • Aeryle88-PGOAeryle88-PGO Posts: 440 ✭✭✭
    Yes, One or More upgrades = Appeal for Several rejection reasons

    I never said my submissions are 100% correct all the time. Sometimes i perfectly understand the rejection and that's end here. But there are many rejection that are not justified, of with false rejection reason (like the mushroom sculpture rejected as a natural feature) and every time that's happen, i post a message to try to understand why some reviewer reject the submission.

    But when i'm sure the submission is correct, i except it to be validated. And if not, i think we should have the opportunity to make an appeal. (or to resubmit it without having to wait 14 days and loose a submission slot.)

    I don't think my nominations are the best. But i think they are good because i took time to check if they meet criteria before submitting them. I don't expect to have 100% of them validated, but if one of them are rejected, i expect to have a real rejection reason, not a false rejection reason.

    Curiously, rejected happen everytime on submissions that i consider to be really good one. For example i submit an old playgroud. That's eligible but it's clearly not the best place of the village. It was accepted at the first submission. The mushroom sculpture i submit two times and rejected two time is on a main palce of the village, it's a really nice place and certainly one of the best potential POI of the village. That's not logical.

  • PlasmonicH2O-PGOPlasmonicH2O-PGO Posts: 3 ✭✭

    I would like to appeal live animal

  • KwyjiboHan-PGOKwyjiboHan-PGO Posts: 128 ✭✭✭
    edited April 2021
    No

    No, instead of placing more work on submitters, Niantic should increase their pre-screening of nominations to introduce more honeypots.

    For many reviewers, I suspect reviewing has become a mechanical process with no critical faculties involved. So postboxes, stinkpipes, children's play areas get 5*s and everything else is rejected. They would still have a great rating.

    We need to shake them out of their stupor.

  • sogNinjaman-INGsogNinjaman-ING Posts: 3,313 ✭✭✭✭✭
    No

    TBH, I don't review nominations to get upgrades. I have 10 already waiting to go and getting a 1% increase is not exactly an incentive. At the moment my running "7-day average" for "correct" reviews is 250-300 per week, so I am already generating 3 potential upgrades per week and have been up as high as 1000 per week in thr past when going for my Platinum OPR..

Sign In or Register to comment.