Feature request: Remove 20 second wait for All-Star reviewers
I have reviewed more than 30,000 nominations and I have more than 23,000 agreements. I was involved in the Operation Portal Recon beta program that rolled out to level 16 Ingress players who were active in Texas (sometime in 2016). I was one of the 20 OPR beta reviewers to get an OPR t-shirt from Niantic for being a great beta tester. I like to think I know what I'm doing in OPR and Wayfarer.
Currently, Wayfarer requires reviewers to wait 20 seconds between reviews. If reviewers review faster, they get a four-hour cooldown after approximately five fast reviews. Reviewers like me can review much faster than 20 seconds. It is quite painful to have to wait 20 seconds on a nomination that is clearly 1-star or a duplicate. Personally, this wait causes me to review less because I get bored waiting 20 seconds and give up for the day.
For reviewers with more than 20,000 agreements and who are in "great" status, can you remove the 20 second wait and just let us review as fast as we're capable? If our rating goes to "good" or lower, the system could reimplement the 20 second wait period until we recover to "great" again. This approach would greatly improve Wayfarer's efficiency while maintaining the same quality.
Comments
Yes, that would be helpful. Since wayfarer nominations got unlocked for lvl 38 PGO players, the quality of the reviewed has massively decreased. Also, we need a "fake warning button", to get other reviewers warned.
What I would also add to what Luke was saying is that I would not like to get misagreements because of bad reviewers. Of course me too I am learning and learning and I do mistakes as everyone. But sometime is sad to see your percentage of agreements going down despite you are following Niantic rules.
Exactly. You have two options:
-Accept because its a great POI
-Reject because its a faked submission
Most reviewers lose their rating because of correctly mark submissions as fake. I habe seen so many incredible fakes like playgrounds on houses or in PRP in the garden. Very sad, but your voice is not being recognized in any way.
i am co-signing this. @NianticCasey-ING
i am much faster in Reviewing than 20 seconds. infact, the system could be cleared out of the most bad reviews if you would have the "Whales" review IN THEIR OWN TIME.
Thank you.
the point is sometime we give 1 stars only based on something that, despite is clearly wrong, is not the only criteria we should look at.
For instance if something has a very bad picture, clearly you give 1 star, but maybe sometime you could give 2 stars and voting other points as 4 5 stars...
In my case sometime is happening to give 2 stars for a very bad photo but I see that other things are ok... And to do that I spend more than 20 seconds... Maybe that's what Niantic would expect from us.
Well if in the photo there is a face I give 1 star, but when it is blury maybe 2 stars are better than 1 instead to say definitively "bad quality"
@pakoesilvia13-PGO Let's try to keep this thread on-topic.
I'd like to see the data on what tiny part of the player base is over 20,000 Recon; just checking around in my local regions, those with Onyx Recon are prominent and active players, period. To combine Onyx with a Great Rating says a lot about the depth to which a reviewer is attuned to the gently shifting criteria and Guidelines of the portal network. Agents that are in the top echelon in one or more of their stats globally, players with hundreds of millions of lifetime Access Points, players that have set goals like an accepted candidate in every county of their state.
Don't ask me, I just work here, but this sounds like an extremely reasonable idea. You don't get that kind of rating by only voting on soft pitches or with any gaming of the system or by gatekeeping with your voting.
Recently, a lot of private residential property submissions that we can easily tell they are prp - literally a shot of someone’s foot or front door - and we just have to wait half a minute or else we get a cooldown.
on the other side you have to understand, that Niantic wants "Quality". They think, for most people, the time to look at somthing and give a quality vote, should be more then 3 seconds.
So they made it that your rating tanks if you click too fast.
Just a simple calc:
after a long 8 hour day at work, and commute of 1 hour, having spend time in the kitchen and showering:
you have about 1 hour before tv/or going catchum all:
givin 30 seconds/review you would end up with ~50 submissions that you checked out.
Out of these ~50 there will mostly be thing that a majority of reviews has a different point of view, and out of these 50 surely you will only earn 25 to 30 "Points" for the Upgrade (100)
If you do that 4 times a week, you might be able to get an Upgrade. That is 4 hours spent for Niantic.
-> If i would be able to vote like I WANT TO, i surely could reviews 5 things in a minute, as mostly submissions are either PRP, Trash or Trash. I would end up with 5x60= ~300 Things that i checked in that 1 Hour. Out of these i surely would be able to get 1 Upgrade a Day instead of taking 4 Days.
Not a bad thing, and this would surely be an incentive for people to use wayfarer.
Theres nothing more boring than having to wait 20+ seconds to click "~Next"
Niantic needs to introduce a special * for those potent and able Reviewers.
This is an interesting suggestion, thanks for adding it @AndysaurusRex-ING. Thinking through this, I can see it being a benefit for those reviewers who have demonstrated consistent, high quality review sessions. This would need to be tied to some sort of back-end nomination quality grade/score/rating/etc.
But herein lies the challenge: How do you prevent "mindless" reviewing? The delay is intentional, so that you spend sufficient time with a nomination to fully process and consider the quality compared to the criteria and fully process the local color added by the nominator. I hear what you're saying about wanting to process more reviews but there's a balancing point between reviewing expediently and reviewing carelessly that we're trying to find.
I'm not saying no, in fact I've surfaced this idea with the Wayfarer Team to consider how we might implement this in the future, but there are some intentions behind this delay.
This is probably a good metric for detecting really terrible submissions. If lots of reviewers in the Onyx+Great category are really quick to reject a submission, it's likely to be totally inappropriate 'coal' that the submitter should be warned/punished for.
Awarding a cooldown to the submitter not the reviewers in such cases would probably be a good idea.
I've recently been through a phase where every time I review more than about 10 in one go I get a cool down, now it all makes sense.
I've always been a fast reader, and often submissions have hardly anything to read or check - sometimes they are the only thing on the duplicate screen for example because the area is so sparse, the title and description are only one or two words, etc so it really doesn't take all that long to read them, whether they are good or bad. Some things are just obviously terrible, and some are obviously eligible.
It definitely makes me not want to review much when I can't go at my own pace and get a cooldown all the time.
well basicly it is all about finding those trustworthy reviewers.
as i said here repeatedly, the system needs sort of patros that pre-select things. if 2 persons kick out a "Priv.Property" Submission or Trash, the system does NOT need to give it to <Amount XY> other viewers that say the same.
If those 2 or 3 Patrons have said "outta here" then it should be rejected.
Niantic needs to speed up the system, and have more quality.
I don't know that this needs to be tied to seniority. I think the Wayfarer Team just needs to adjust their expectations such that all reviewers are allowed to reject some nominations quickly (so long as many or most of their reviews still last an acceptable amount of time). I really don't need to spend 30 seconds to reject a photo of someone's TV, but still, most reviews should take 30 second or more.
@NianticCasey-ING You don't need to prevent mindless reviewing in people who have a solid history of high-quality reviews-- those people have already proven that they do not review mindlessly. There are plenty of things that I can look at and immediately reject based on depth and breadth of experience, and also a firm grasp of the obvious. "Stop sign". 1*, doesn't meet criteria, next. "My front door", 1*, private residence or farm, next. "Bob Smith Junior High School", 1*, K-12, next. I say this as someone who has the onyx recon badge and over 16K total reviews.
I obviously don't have access to mine Niantic's data to identify the set of criteria that would qualify someone as a highly-competent reviewer but I imagine it would be some combination of length of time reviewing (maybe one year minimum), number of agreements (5K or 10K minimum maybe), agreement rate percentile (top 10% or 20% perhaps), and some heuristics for agreement distribution for things. That last one is the hardest one because you need to weed out local cabals and people who tend to either approve or reject everything. The cabals are likely to have a very high agreement rate within their group and a much lower one with non-local players, and the always-approve/always-disapprove people will obviously have a very skewed distribution of approval/rejection.
I've done 24k reviews, have double Onyx Recon and have discovered almost 600 portals. Reviewing is inherently mindless, especially when 90% of the things currently in queue are absolute trash and instant rejections. I can determine the quality of a review in a matter of seconds and finalise the review within 10 seconds. I look at the forums every day and know the guidelines inside and out. Prior to the cooldowns I could do 100, 200+ reviews in a single session / a day, now I struggle to do 30+ without getting a cooldown. The system is too punishing and it has driven away so many people.
It shouldn't be up to the reviewers to constantly nag about the issues, Niantic should be proactive on actually streamlining the process and addressing reviewer's concerns, yet we are constantly fallen onto deaf ears and now look at where we at - the whole thing is a mess.
As somone with over 40k reviews, I have to agree that the 20 second wait "or you get a cooldown" is very frustrating. Its not "mindless reviewing", but I'm sure most reviewers can spot "coal", or a PRP, or a Scout Hut, or a Primary School, or an unnamed generic "trail marker" within a couple of seconds. Some reviews do need some time, particularly when you are trying to confirm the location from Streetview and the supporting photos, or you have to check links or Google to see if the site is really what is described, or a fake trying to squeeze in, but overall, if you want the backlog reduced, take off the brakes please.
I reviewed more than 140K and I am a user with 100K Recon medals.
It only takes a little longer than before, but more than 200 reviews a day can still be done.
However, it doesn't seem very good to have to use 20~30 seconds to avoid being cool-downed by a 1* candidate.
What I don't get is that I always gets cooldowns after things that don't make sense like rejecting a mailbox or choosing between two edits or choosing the photo that don't meet criteria. I just don't get it. If one edit has tons of grammar mistakes and the other one does not. It won't take me 20 seconds to notice it.
How exactly does the 20-second rule work? I have done 25k reviews, and I am pretty sure that some of them took less than 20 seconds, even with writing a short comment. I only got one cooldown in 18 months.
theres an amount X, if you have clicked thru under the 20 (i still think it is 30 seconds) seconds, then you will get a thing that Denis Nedry has build in.
Hi @NianticCasey-ING ,
It's been a few months since I surfaced this idea, but I see the cooldown is still there for reviewers who go faster than 20 seconds per review. I'm just wondering if there have been any additional internal discussions about removing the 20 second cooldown for "All Star" Wayfarer reviewers as I suggested in my original post.
As suggested by some people in this thread, it would be great if Niantic could remove speed restrictions for proven Wayfarer reviewers who have more than 20,000 agreements, are rated "great", and have been reviewing for more than a year. This seems like something the Wayfarer development team could manually configure rather quickly.
Using the agent-stats' recon group (https://www.agent-stats.com/groups.php?group=recon) as a guide, I would estimate there are fewer than 100 Ingress players with at least 20,000 agreements. I am not sure how many Pogo players have at least 20,000 agreements, but it would also probably be pretty small. This small group would allow Niantic to use these rather trusted reviewers in a small pilot program to determine if removing the speed restrictions has any negative quality issues with the wayspot network.
20k would be very high, i am at 11.614
@AndysaurusRex-ING I'm in favor of your proposal, but my personal preference would be for the cutoff to be lower... and admittedly that is partially for selfish reasons. (My badge stat is just over 13K, but my actual agreement count is just over 11K.) I've proposed a similar concept before. I too have been reviewing since the OPR beta.
If I was defining criteria for elite reviewers I think it would be:
The benefits I would grant these reviewers:
My rationale for those benefits: If you've reviewed that long and that much and you've kept your rating up you probably know what you are doing and it's easier for you to make decisions quickly. I've gotten the occasional cooldown myself from a string of several "Really, you're kidding, you submitted THAT?!" reviews in succession, and I wind up skipping reviews or taking a break to avoid cooldowns. I could be more productive as a reviewer if I didn't have to dodge hurdles constantly.
Reducing the bonus location time would allow experienced high-quality reviewers to become an underserved-area hit squad. "Oh, that country has few wayspots and needs reviewers? Sure, I'll drop in and help." I'm not willing to be stuck in Far Distant Island Nation for a year, but I'd gleefully go help bootstrap an area for a month or two.
As Marx said, "I'm not interested in being part of any club that would accept me as a member", that of course being Groucho.
So while I won't meet any reasonable requirements for quite some time, I agree with this proposal.
I have a simple solution.
You remove the 20 seconds for the nomination ranked 1* only.
Efficient reviewer to pitch out trash quickly.
Mindless reviewer (I think) want in-games bad portal quickly. I don't think they want to spam the refusal button.
Ah yes also the cooldown for picture reviewing is what is killing me the most. It is litterally 50 % of my cooldowns.
No. Everyone will start to 1* everything. That wont work...
Well, why they would do this ? Real question.
Also, in case of massive rejection this peoples will have extremly low agreement and by then, really low impact.
"Hey, reject everything. If enough players will do the same, you will get agreements in a super short period of time"