Feature request: Remove 20 second wait for All-Star reviewers

AndysaurusRex-INGAndysaurusRex-ING Posts: 5 ✭✭
edited December 2020 in General Discussion

I have reviewed more than 30,000 nominations and I have more than 23,000 agreements. I was involved in the Operation Portal Recon beta program that rolled out to level 16 Ingress players who were active in Texas (sometime in 2016). I was one of the 20 OPR beta reviewers to get an OPR t-shirt from Niantic for being a great beta tester. I like to think I know what I'm doing in OPR and Wayfarer.


Currently, Wayfarer requires reviewers to wait 20 seconds between reviews. If reviewers review faster, they get a four-hour cooldown after approximately five fast reviews. Reviewers like me can review much faster than 20 seconds. It is quite painful to have to wait 20 seconds on a nomination that is clearly 1-star or a duplicate. Personally, this wait causes me to review less because I get bored waiting 20 seconds and give up for the day.


For reviewers with more than 20,000 agreements and who are in "great" status, can you remove the 20 second wait and just let us review as fast as we're capable? If our rating goes to "good" or lower, the system could reimplement the 20 second wait period until we recover to "great" again. This approach would greatly improve Wayfarer's efficiency while maintaining the same quality.

Post edited by NianticCasey-ING on
Tagged:
«1

Comments

  • pakoesilvia13-PGOpakoesilvia13-PGO Posts: 127 ✭✭

    What I would also add to what Luke was saying is that I would not like to get misagreements because of bad reviewers. Of course me too I am learning and learning and I do mistakes as everyone. But sometime is sad to see your percentage of agreements going down despite you are following Niantic rules.



  • LukeAllStars-INGLukeAllStars-ING Posts: 4,625 Ambassador

    Exactly. You have two options:

    -Accept because its a great POI

    -Reject because its a faked submission


    Most reviewers lose their rating because of correctly mark submissions as fake. I habe seen so many incredible fakes like playgrounds on houses or in PRP in the garden. Very sad, but your voice is not being recognized in any way.

  • pakoesilvia13-PGOpakoesilvia13-PGO Posts: 127 ✭✭

    the point is sometime we give 1 stars only based on something that, despite is clearly wrong, is not the only criteria we should look at.

    For instance if something has a very bad picture, clearly you give 1 star, but maybe sometime you could give 2 stars and voting other points as 4 5 stars...

    In my case sometime is happening to give 2 stars for a very bad photo but I see that other things are ok... And to do that I spend more than 20 seconds... Maybe that's what Niantic would expect from us.

    Well if in the photo there is a face I give 1 star, but when it is blury maybe 2 stars are better than 1 instead to say definitively "bad quality"

  • @pakoesilvia13-PGO Let's try to keep this thread on-topic.

  • Strikew3st-INGStrikew3st-ING Posts: 1 ✭✭

    I'd like to see the data on what tiny part of the player base is over 20,000 Recon; just checking around in my local regions, those with Onyx Recon are prominent and active players, period. To combine Onyx with a Great Rating says a lot about the depth to which a reviewer is attuned to the gently shifting criteria and Guidelines of the portal network. Agents that are in the top echelon in one or more of their stats globally, players with hundreds of millions of lifetime Access Points, players that have set goals like an accepted candidate in every county of their state.

    Don't ask me, I just work here, but this sounds like an extremely reasonable idea. You don't get that kind of rating by only voting on soft pitches or with any gaming of the system or by gatekeeping with your voting.

  • AbinitioZ-PGOAbinitioZ-PGO Posts: 34 ✭✭✭

    Recently, a lot of private residential property submissions that we can easily tell they are prp - literally a shot of someone’s foot or front door - and we just have to wait half a minute or else we get a cooldown.

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    on the other side you have to understand, that Niantic wants "Quality". They think, for most people, the time to look at somthing and give a quality vote, should be more then 3 seconds.

    So they made it that your rating tanks if you click too fast.

    Just a simple calc:

    after a long 8 hour day at work, and commute of 1 hour, having spend time in the kitchen and showering:

    you have about 1 hour before tv/or going catchum all:

    givin 30 seconds/review you would end up with ~50 submissions that you checked out.

    Out of these ~50 there will mostly be thing that a majority of reviews has a different point of view, and out of these 50 surely you will only earn 25 to 30 "Points" for the Upgrade (100)

    If you do that 4 times a week, you might be able to get an Upgrade. That is 4 hours spent for Niantic.

    -> If i would be able to vote like I WANT TO, i surely could reviews 5 things in a minute, as mostly submissions are either PRP, Trash or Trash. I would end up with 5x60= ~300 Things that i checked in that 1 Hour. Out of these i surely would be able to get 1 Upgrade a Day instead of taking 4 Days.

    Not a bad thing, and this would surely be an incentive for people to use wayfarer.

    Theres nothing more boring than having to wait 20+ seconds to click "~Next"

    Niantic needs to introduce a special * for those potent and able Reviewers.

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    well basicly it is all about finding those trustworthy reviewers.

    as i said here repeatedly, the system needs sort of patros that pre-select things. if 2 persons kick out a "Priv.Property" Submission or Trash, the system does NOT need to give it to <Amount XY> other viewers that say the same.

    If those 2 or 3 Patrons have said "outta here" then it should be rejected.

    Niantic needs to speed up the system, and have more quality.

  • flatmatt-PGOflatmatt-PGO Posts: 2,141 ✭✭✭✭✭
    edited December 2020

    I don't know that this needs to be tied to seniority. I think the Wayfarer Team just needs to adjust their expectations such that all reviewers are allowed to reject some nominations quickly (so long as many or most of their reviews still last an acceptable amount of time). I really don't need to spend 30 seconds to reject a photo of someone's TV, but still, most reviews should take 30 second or more.

  • Hosette-INGHosette-ING Posts: 3,470 ✭✭✭✭✭

    @NianticCasey-ING You don't need to prevent mindless reviewing in people who have a solid history of high-quality reviews-- those people have already proven that they do not review mindlessly. There are plenty of things that I can look at and immediately reject based on depth and breadth of experience, and also a firm grasp of the obvious. "Stop sign". 1*, doesn't meet criteria, next. "My front door", 1*, private residence or farm, next. "Bob Smith Junior High School", 1*, K-12, next. I say this as someone who has the onyx recon badge and over 16K total reviews.

    I obviously don't have access to mine Niantic's data to identify the set of criteria that would qualify someone as a highly-competent reviewer but I imagine it would be some combination of length of time reviewing (maybe one year minimum), number of agreements (5K or 10K minimum maybe), agreement rate percentile (top 10% or 20% perhaps), and some heuristics for agreement distribution for things. That last one is the hardest one because you need to weed out local cabals and people who tend to either approve or reject everything. The cabals are likely to have a very high agreement rate within their group and a much lower one with non-local players, and the always-approve/always-disapprove people will obviously have a very skewed distribution of approval/rejection.

  • HINABITAMEU-INGHINABITAMEU-ING Posts: 154 ✭✭✭

    I reviewed more than 140K and I am a user with 100K Recon medals.

    It only takes a little longer than before, but more than 200 reviews a day can still be done.

    However, it doesn't seem very good to have to use 20~30 seconds to avoid being cool-downed by a 1* candidate.

  • Euthanasio2-PGOEuthanasio2-PGO Posts: 272 ✭✭✭

    What I don't get is that I always gets cooldowns after things that don't make sense like rejecting a mailbox or choosing between two edits or choosing the photo that don't meet criteria. I just don't get it. If one edit has tons of grammar mistakes and the other one does not. It won't take me 20 seconds to notice it.

  • saarstahl-INGsaarstahl-ING Posts: 184 ✭✭✭

    How exactly does the 20-second rule work? I have done 25k reviews, and I am pretty sure that some of them took less than 20 seconds, even with writing a short comment. I only got one cooldown in 18 months.

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    theres an amount X, if you have clicked thru under the 20 (i still think it is 30 seconds) seconds, then you will get a thing that Denis Nedry has build in.

  • AndysaurusRex-INGAndysaurusRex-ING Posts: 5 ✭✭

    Hi @NianticCasey-ING ,

    It's been a few months since I surfaced this idea, but I see the cooldown is still there for reviewers who go faster than 20 seconds per review. I'm just wondering if there have been any additional internal discussions about removing the 20 second cooldown for "All Star" Wayfarer reviewers as I suggested in my original post.

    As suggested by some people in this thread, it would be great if Niantic could remove speed restrictions for proven Wayfarer reviewers who have more than 20,000 agreements, are rated "great", and have been reviewing for more than a year. This seems like something the Wayfarer development team could manually configure rather quickly.

    Using the agent-stats' recon group (https://www.agent-stats.com/groups.php?group=recon) as a guide, I would estimate there are fewer than 100 Ingress players with at least 20,000 agreements. I am not sure how many Pogo players have at least 20,000 agreements, but it would also probably be pretty small. This small group would allow Niantic to use these rather trusted reviewers in a small pilot program to determine if removing the speed restrictions has any negative quality issues with the wayspot network.

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    20k would be very high, i am at 11.614

  • Hosette-INGHosette-ING Posts: 3,470 ✭✭✭✭✭

    @AndysaurusRex-ING I'm in favor of your proposal, but my personal preference would be for the cutoff to be lower... and admittedly that is partially for selfish reasons. (My badge stat is just over 13K, but my actual agreement count is just over 11K.) I've proposed a similar concept before. I too have been reviewing since the OPR beta.

    If I was defining criteria for elite reviewers I think it would be:

    • 5K agreements
    • Great rating maintained for at least, oh, six months
    • One year of reviewing

    The benefits I would grant these reviewers:

    • Significantly higher tolerance for cooldowns
    • More frequent bonus location changes

    My rationale for those benefits: If you've reviewed that long and that much and you've kept your rating up you probably know what you are doing and it's easier for you to make decisions quickly. I've gotten the occasional cooldown myself from a string of several "Really, you're kidding, you submitted THAT?!" reviews in succession, and I wind up skipping reviews or taking a break to avoid cooldowns. I could be more productive as a reviewer if I didn't have to dodge hurdles constantly.

    Reducing the bonus location time would allow experienced high-quality reviewers to become an underserved-area hit squad. "Oh, that country has few wayspots and needs reviewers? Sure, I'll drop in and help." I'm not willing to be stuck in Far Distant Island Nation for a year, but I'd gleefully go help bootstrap an area for a month or two.

  • Lechu1730-PGOLechu1730-PGO Posts: 537 ✭✭✭✭

    As Marx said, "I'm not interested in being part of any club that would accept me as a member", that of course being Groucho.

    So while I won't meet any reasonable requirements for quite some time, I agree with this proposal.

  • Cdk296-PGOCdk296-PGO Posts: 170 ✭✭✭

    I have a simple solution.

    You remove the 20 seconds for the nomination ranked 1* only.

    Efficient reviewer to pitch out trash quickly.

    Mindless reviewer (I think) want in-games bad portal quickly. I don't think they want to spam the refusal button.

  • Cdk296-PGOCdk296-PGO Posts: 170 ✭✭✭

    Ah yes also the cooldown for picture reviewing is what is killing me the most. It is litterally 50 % of my cooldowns.

  • Cdk296-PGOCdk296-PGO Posts: 170 ✭✭✭

    Well, why they would do this ? Real question.

    Also, in case of massive rejection this peoples will have extremly low agreement and by then, really low impact.

  • LukeAllStars-INGLukeAllStars-ING Posts: 4,625 Ambassador

    "Hey, reject everything. If enough players will do the same, you will get agreements in a super short period of time"

Sign In or Register to comment.