What to do about "reject to get agreement" reviewers?
cyndiepooh-ING
Posts: 1,324 ✭✭✭✭✭
Saw this posted on reddit:
"The biggest sin of the system is that it perversely incentivizes agreements. You're actively discouraged from careful evaluation and encouraged to try to figure out what the hivemind will think. Add this to the estimate that it takes about 45 4-5 star votes to get an acceptance, and only 5 one-stars to get a rejection and you get an inherently awful system.
Heck, try it yourself, try doing careful and consistent review vs at random rejecting 2 submissions and only evaluating the 3rd one. Guarantee your rank in Wayfarer will go up and you will start getting agreements up the wazoo.
I had some discussions where I expressed my frustration with the system to veterans, and their reply was "Oh, you need to just reject more." I thought that was stupid, but after trying it out I can say they were 100% right."
So no wonder we are seeing so many incorrect and random rejections if people are trying to game agreements through rejections =(
Comments
Where are you getting the 45 for agreement 5 for rejection statistic? Cause there's nothing to support that at all other than a poorly done test by some guy on reddit with added speculation.
I reveiw a lot of nominations and only very very rarely give a "5*" acceptance, my default for a "standard - no problems waypoint" is 3*. I save 5* for something that really deserves to be made a Waypoint.
It’s funny, when word of nominations and reviews first started trickling into the Pogo social ether, a friend of mine asked, “Why would anyone reject? Who doesn’t want more Pokestops?”
And now we read ‘Why vote to approve? The system rewards rejections.’
3 star is essentially a skip from what I have seen and read. It's essentially "don't know". 4 star is like the minimum acceptance
The irony being, getting an upgrade through only rejecting means your upgraded non will most likely get rejected
I have seen a few posts/comments about people reviewing this way. It's always been anecdotal--"someone I know says they do this," "I've heard that this is how people review to get agreements," etc. I haven't seen a post from someone saying that they reject everything.
I don't doubt it's possible, but I don't know if it's really as big a problem as it's being made out to be.
Oh, you need to just reject more."
This surely helps explain why so many should-be-shoe-in nominations like trail markers and group activities keep getting rejected. Gah.
oh that was what was being said - not my information. would be great to have someone from niantic squash that rumor officially.
i hope you are right
The 50 is coming from people on the Silph Road Reddit are quoting this https://www.reddit.com/r/NianticWayfarer/comments/mcw1ju/research_how_many_people_evaluate_a_proposal_and/
in the context of this thread: https://www.reddit.com/r/TheSilphRoad/comments/pxoxk1/niantic_recently_published_a_pretty_simplified/
So you’re saying you see very little that you deem deserves to be a waypoint? 🤔
I will 5* anything that I think should pass thru. 3* if I’m unsure. I only will 1* for obvious abuse or garbage.
I do think this is a trend. I do know of at least one person that rejects way more than I think they should be. They take the criteria very literally and will not diverge from that, even if consecutive updates say it’s ok. And it’s pretty obvious others do this as well when people are having stuff that should be easy passes (parks, churches, artwork, etc) get rejected.
So they're saying they were able to move a wayspot after rievewing it with 5 accounts, (without knowing what the other reviewers did) so they make the leap that you can reject one with 5 accounts. And they're basing the 50 number off of google maps views, something that anyone can access.
That is a really bad study and conclusion TBH. I've have photospheres posted for nominations get less than 30 views before my wayspot was approved.
And of course they share no details of the submissions themselves so we don't know anything about how that could have had any effect on outcomes.
Well, I've for over 14k "acceptances" on my stats scoring nominations as "3*" for a "standard" Waypoint, so I don't think a 3* = a "skip".
we don’t know how voting works but it would be logical for it to have a points basis related to stars.
So you might expect there to be a points total to be reached and that if say 20 people give 5* then it gets to 100 and passes but if more people just give 3* it’s going to need 34 people to get 100. I’m not suggesting those are the numbers but the effect of 3* means that the flow of reaching agreement slows and if there is a lack of reviewers it can grind to a halt.
By giving a 3* an individual does not lose out on an agreement - it will come eventually - but too many doing this instead of scoring higher slows the system.
That is a really bad study and conclusion TBH. I've have photospheres posted for nominations get less than 30 views before my wayspot was approved.
Same here. I’ve even had 20-25 reviews between when something went into voting and when it was approved. These were typically nice trail markers in very isolated areas.
I'm going by what niantic have said, that 3 star is maybe/don't know. If you think something should pass, 4 star is supposed to be the acceptable, with 5 star being good to excellent, and it's what the vast majority of reviewers think as well judging by reddit and this forum
Might be other factors as well. We k ow having great and good ratings count more than fair and poor (from niantic telling us I believe, might be wrong on that). I also think, though I obviously can't be sure, that local reviewers, so people reviewing stuff in the area they play and not necessarily the cells, might count for less to avoid cabals just accepting everything, though, the amount of fake wayspots through suggests otherwise, so might be the opposite local counts for more.
I think a general consensus thoigh is tjat it does take less rejects to reject something than it does to pass. Amd it would also account for some dot he bizarre rejection reasons, I don't see many people selecting live animal for a park, or submitter identifiable for a church, but have seen people get those rejection reasons for those exact things
Wayspots within Large Areas
Wayspots within large areas like parks, plazas, and fields are eligible as well, even if the larger area itself is a Wayspot. Just keep in mind that someone may have already made a nomination on various objects/placemarkers around entrances, which could lead to a rejection (as a duplicate).
For the "overall" Nomination review score, where have Niantic said this?
Good enough?
I have seen a large number of people say something like, "I think this is a great nomination, but non-locals won't vote to approve restaurant x so I rejected it." They voted in order to get an agreement rather than based on the actual reviewing criteria, and those individuals aren't alone. Usually, it's only once every month or so, but I find it frustrating. People are primarily interested in helping themselves rather than a sense of helping other people or desire to improve the quality of Niantic's database.