Penalizing Rejection

I do not know if this is a thing... When a submission gets rejected for a wrong reason, we could appeal it. After the appeal process, the nomination may get accepted ot rejected. When the nomination gets accepted after the appeal... do the accounts that rejected get penalized in any way .... getting rejections that show that the reviewer does not understand the acceptance and rejection criteria is frustrating... repeated acts from bad reviewers should be penalized in my opinion.

Please Note... I'm talking about just reading

For example,

K-12 rejection > It is a university or college

Pedestrian access > Public park

Something really simple

Those type of rejections that show the reviewer does not read.

Comments

  • Hosette-INGHosette-ING Posts: 3,424 ✭✭✭✭✭

    One of the things that happens quite frequently is that the rejection itself was correct but the reasons that Niantic displays to the submitter don't fit the situation. This happens so often that I think a lot of the problem is the way that Niantic chooses the rejection reasons to display rather than what reviewers do. The new review workflow changes this some, and we don't yet know what the impact will be.

    It takes a majority of reviewers to reject something (we believe), but it may be the case that it only takes one single reviewer choosing a rejection reason for it to be displayed to the submitter. If you start penalizing reviewers because they accidentally clicked the wrong reason then you're going to lose a lot of reviewers.

    Sometimes things are because "the reviewer does not read" but just as often it's because the submitter didn't provide enough information for the information to be obvious. There are things in public parks that don't have safe pedestrian access, like fountains in the middle of ponds.

  • The26thDoctor-PGOThe26thDoctor-PGO Posts: 4,636 ✭✭✭✭✭

    Which sort of penalty would you like to see handed out?

  • mrr5745-PGOmrr5745-PGO Posts: 11 ✭✭✭

    Niantic rejected a permanent art installation at an international airport. The nomination included a link to a website proving it exists and was permanent. I officially give up.

  • Hosette-INGHosette-ING Posts: 3,424 ✭✭✭✭✭

    @FireboyX-PGO Do you expect reviewers to be 100% accurate in everything they do, even over thousands of reviews? Do you really expect that level of accuracy from unpaid volunteers? Just like submitters, reviewers who make large systemic errors should be corrected. Reviewers who make an occasional error should not.

    Submitters should submit valid things but they often don't. Should they be punished every time something of theirs is correctly rejected?

  • FireboyX-PGOFireboyX-PGO Posts: 8 ✭✭

    @Hosette-ING Do you expect reviewers to be 100% accurate .... To some degree YES !... If you want the reviewers who accept the nominations to be accurate, you want them to reject with the same accuracy. I do not want the community who spend time to explore and nominate wayspots ... for their time to be wasted. Yes.. we can appeal, but as I said... The mentality of I will submit this to be rejected by (malicious or uneducated) reviewers... then let Niantic review it through their appeal system is not good....

    Think please ... If all your nominations, with whatever you wrote and evidence gets rejected everytime

    .. and appeal always work, you know there are a group of people in your community that are maliciously rejecting the nominations. Of course, that is a problem specifically in my area... as far as I know.

    You make the idea that reviewers are unpaid ... but also submitters are unpaid too. Submission now have the abuse catagory, reviewers can be malicious too and abuse the rejection and the acceptance of wayspot nominations.

    If the rejection actually need majority of reviewers to reject, then appeal said otherwise, that in itself speaks volumes. Because in no way you can expect the majority of reviewers to be wrong about the rejection criteria and acceptance criteria. I'm sick of K-12 rejections that are in university and college grounds, all universities have a website, and their location in Google Maps / Satalite view is 100% clear. you can not possibly reject K-12 wrongly by a misclick , you need multiple presses to reach that point.

    I'm personally planning into going into a brand new residential area 30 mins drive away from me to nominate churches , malls, parks , playgrounds, and whatever fits the acceptance criteria, but the idea that spending fuel to go there and spending probably 3 hours roaming to find appropriate nominations for it to end at 80% rejection... That's a demotivating idea. As my country have 1000s of projects that make 1000s of nominations... Malicious rejectors are in the way. We need a solution... my solution is penalty after appeal.... Do you have a better idea ?!

  • FireboyX-PGOFireboyX-PGO Posts: 8 ✭✭

    Redo the Wayfarer quiz

    It repeated multiple times , a timed ban from wayfarer....

    I'm talking about reviewers who either do not understand or read all the information given, or honestly deliberate rejection for hating.


  • Hosette-INGHosette-ING Posts: 3,424 ✭✭✭✭✭

    @FireboyX-PGO Reviewers will never be 100% perfect. Humans are never 100% perfect at anything.

    To a large extent the problem is handled by the wisdom of crowds. If 40 people review a submission and one or two of them make a mistake then it's really not a big deal because they won't affect the consensus. The issue of "wrong" rejection reasons only arises in cases where the consensus of all the reviewers was to reject the submission, and more often than not when people complain about bad rejection reasons the rejection itself was legitimate even if some of the specified reasons didn't make sense. Under the old system there were several times when I meant to hit "Other rejection criteria" and accidentally clicked "Live animal" instead. I've caught myself doing this several times before I hit submit (over the course of more than 30K reviews), and it's entirely possible that sometimes I didn't catch the error. My conclusion that the submission should be rejected wouldn't have changed, and thus accidentally choosing the wrong rejection reason wouldn't have affected the outcome... it just would have given the submitter a reason that didn't make sense.

    We don't know how much noise there is in the system but it's certainly not zero. It's seems likely that most submissions have at least one no vote even when they're accepted but you'd never know that because the consensus was to accept. It's also likely that most rejected submissions have at least one vote to accept.

    It is certainly possible to have voting cabals deliberately making bad decisions... we saw this in the Netherlands recently with bots who accepted everything. It's also common for local areas to have imagined their own set of rules rather than the ones Niantic provides for us... this happens when a few things slip through that should have been rejected (e.g. fountains in the middle of ponds, subdivision entrance signs) and then the people in the area base their ideas of what should be accepted on what they see in the games rather than the rules that Niantic has laid out for us. Malicious and uneducated reviewers are totally different situations from the cases where someone just clicked the wrong reason when they voted to reject.

  • The26thDoctor-PGOThe26thDoctor-PGO Posts: 4,636 ✭✭✭✭✭

    You still haven't said which type of penalty you would like bad reviewers to receive? @FireboyX-PGO

  • Leedle95-PGOLeedle95-PGO Posts: 605 ✭✭✭✭

    In many parts of the world, colleges are secondary schools, not post-secondary, as they are in some countries. If you had reviewers from outside your country, that may be part of the problem. Even if the reviewers properly understood the definition where you are, I can see it very plausible that they wouldn’t find that particular nomination eligible as a great place to exercise, explore, or socialize.

    Also, I have seen, as have others, many things getting through on appeals that were correctly rejected. This is probably the most frustrating because the community decides correctly based on Niantic guidance, then someone in the group doing appeals gets it wrong.

    I agree with the other commentators, the whole issue is far more nuanced. There may be some POI that are clearly eligilbe, but I would say more times than not the POI that is submitted may be eligible, and it’s up to the person, submitting it to be convincing, and that’s where there is often a gap, leaving it up to the community who reviews to interpret and decide.

    Yes, people who are intentionally reviewing poorly to get at some goal need to be made aware that that is inappropriate, however, it may be very difficult to disentangle those who are poorly reviewing systematically from those who simply disagree with the person who nominates the POI, or the nominator simply just didn’t provide enough detailed information for reviewers to understand what they had intended. I don’t think the example of what you posted above is as clear cut as you might think.

  • Shilfiell-INGShilfiell-ING Posts: 1,544 ✭✭✭✭✭

    Penalties for rejections that Niantic overturns come in the form of "educational" emails that fail to educate, and generally just scare reviewers away - "you're doing it wrong, keep it up and we'll ban you, re-read the criteria and don't violate again" sort of language. If reviewers who reject a successfully-appealed candidate are thus "punished", what sort of consequence do reviewers who accept candidates on PRP, school grounds, highway medians, etc. receive when those Wayspots are removed based on Wayfarer reports? Do they at least get an agreement subtracted from their record, and do the people who originally rejected the invalid candidate get an agreement added?

  • cyndiepooh-INGcyndiepooh-ING Posts: 1,234 ✭✭✭✭✭

    Since more information can be added to an appeal than original reviewers saw, I do not think it is fair to take away an agreement from reviewers who voted with the majority to reject it. I do not believe Niantic has the time and manpower to comb through and decide which rejections were not correct for the nomination as originally presented. Since we are now dealing with a completely different way to review, punishing people for decisions made under the old system would not be fair.

  • mokou1234-INGmokou1234-ING Posts: 99 ✭✭

    審査における虐待的な否認による不正行為も虚偽申請や承認リングと同様の罰則とする必要はあると思います。

    不正申請や承認リング行為と同様にストライクポリシーのカウントとして加えるべきです。

    ゲームにおける優位性確保の為の虐待的な承認行為が罰せられるにも関わらず、同じ目的による虐待的な否認行為が罰せられないのは馬鹿げています。

    Nianticは虐待的な否認によって優位性が確保される人々がいることを認識するべきです。

    必ずしもWayspotが増えることだけがゲームの有利に繋がるわけではありません。

    例として、Ingressではテーマパーク内等の制限付きエリアはポータルを奪うことが難しく、Ingressプレイヤーはこれをなんとかして否認しようという傾向があります。

    またポケモンGOにおいては、自宅周辺にジムを作成する為に他者の申請を拒否して承認される順番を操作する者もいます。

    このような私的で身勝手な理由で有効な候補がWayspotとして登録されないことはNianticのマッピングサービスの品質を貶め、将来的な顧客を失うことになります。

  • holdthebeer-INGholdthebeer-ING Posts: 190 ✭✭✭✭

    Here's one idea.. since most of "reviewers" are also "submitters", what if nominations submitted by "bad reviewers" (reviewers who misclick rejection reasons, do not read text, and accept garbage) were sent to bad reviewers too, with education message of course? And if those bad reviewers have poor rating, it will also take more bad reviewers to reject nomination, but they will at least try to use correct reasons.

    Another idea is to display ads in wayfarer to bad reviewers. /jk

  • The26thDoctor-PGOThe26thDoctor-PGO Posts: 4,636 ✭✭✭✭✭

    If a bad reviewer reviews a bad submission they would accept it since they are reviewing badly?

  • Hosette-INGHosette-ING Posts: 3,424 ✭✭✭✭✭

    @cyndiepooh-ING Not only can information be added by the submitter when something is appealed but the underlying review conditions can change. I've had to reject quite a few things over the years because they didn't appear in street or satellite view when they were brand new and the submitter didn't provide enough information for me to confirm the location. If the submission was appealed a few weeks or a few months later then the candidate may now be visible on street/satellite view and be easily accepted.

    Thus it's possible for the original rejection to be 100% correct and also for the acceptance on appeal to be 100% correct.

    I will add that a lot of the problem of bad individual reviewers is handled by the great/good/fair/poor system. Reviewers who are in fair or poor status don't count as much toward the decision as those with higher ratings. (Unfortunately, when an area colludes to review badly this system reverses itself.)

  • Ochemist-INGOchemist-ING Posts: 355 ✭✭✭✭✭

    Thus it's possible for the original rejection to be 100% correct and also for the acceptance on appeal to be 100% correct.

    Fully agree with this, and to add to the point, criteria change. For example, at one point, we were told that trail markers had to have the name of the trail on them. I (correctly) rejected some at the time that were just mile markers. Upon appeal now, those would probably be (correctly) accepted, again making both the initial rejection and appeal acceptance 100% correct. Would it be fair to punish me for following the current guidelines while I did the review? I'd probably be completely done with reviewing if that happened.

Sign In or Register to comment.