Newbie reviewer tips

Hi,

I've been trying to review submissions but I must not fully understand how to rate each submission as I'm now at less that 50% acceptance. I've reviewed over 1000 nominations, but my scores are Accept/Reject of 108/111. Having plaid PoGO since day one, I'm pretty sure I know what a good Pokestop should be, but I guess acceptance rules have changed over time.

So some questions:

  1. Those Accept/Reject totals mean that my nominations have gone for further review by Niantic or other reviewers?
  2. How do I score a nomination? 1 star = Reject, but what about 2, 3, 4,5 stars? Does 2 stars mean it gets nominated further?
  3. Does each criteria rating get added up and they have to meet a minimum % to be accepted?
  4. How do promotions work?

Is there an FAQ or a tips page?

Thanks

David

Comments

  • Hosette-INGHosette-ING Posts: 3,469 ✭✭✭✭✭

    Niantic doesn't give us many details about how the stars affect the outcome. I believe that what you say in #3 is sort of accurate, but that different fields have more or less weight. "Should this be a Wayspot?" has the most weight.

    If you add up Nominations Accepted/Rejected/Duplicated you get a number that equals your total agreements. If you subtract that from Total Nominations Reviewed what's left contains two different categories of things-- reviews where you disagreed with the consensus of reviewers and things that are still in review by other players and no decision has been made yet. In a recent experiment, I learned that 50% of the things I review will resolve by around four weeks and 90% will resolve within ten weeks. Thus, things can be in limbo for a long time before there's a consensus.

    My 2021 data are here: https://community.wayfarer.nianticlabs.com/discussion/21901/2021-data-how-long-does-it-take-for-things-youve-reviewed-to-reach-a-decision

  • niktero-PGOniktero-PGO Posts: 369 ✭✭✭✭✭
    1. You gain an agreement (accept/reject/duplicate) when the nomination you voted on gets decided and the majority of people voted the same as you. Lots of people need to vote on one thing before a decision is reached to this can take a while if you were the first voter, or it can happen instantly if you were the last voter needed.
    2. 1 star in any category is likely to cause the whole nomination to be rejected. This was asked in detail in an AMA and Niantic denied that this was going on, but we have few other ideas for how people get rejections with only "not culturally significant" listed. 2 stars means you think something is very wrong and do not want to accept but are not positive. 3 stars is neutral. 4 stars if for accept but this is not perfect and I need to vary my voting patterns. 5 stars is "this is great!" nice photo/great grammar/locally significant/easy to find/ visible on sat or streetview.
    3. We are not sure exactly how the different categories are added but know that rating below a 3 in some categories can cause a rejection.
    4. Promotions/ Upgrades are resolved faster than normal submissions. Normal ones are only seen by people within a certain distance of you but upgraded ones can be seen by anyone in your nation and all it's territories. This means a lot more people vote on it in a much shorter amount of time. The problem is that many of these people are far enough away that they may not know why a certain store, or area is important for you. It is best to use upgrades on things that everyone knows and can relate to like parks, religion, sports, art, or playgrounds.
  • Hosette-INGHosette-ING Posts: 3,469 ✭✭✭✭✭

    @niktero-PGO Your #2 and #3 are false. Niantic has clearly and unambiguously stated that they are false. We have at least one solid theory to explain the rejection reasons.

  • PlutoIsSad-PGOPlutoIsSad-PGO Posts: 188 ✭✭✭

    I've never had anything less than "great" in almost 5,000 reviews. I used to be strict with my number choice in reviews (but still had great) though reading the forums realized that I was still shorting many nominations.

    My own general rating system now for wayspot nominations I think are good:

    4 for should it be a wayspot (5 if it's awesome)

    4 for description if it's fine (5 if it's awesome.)

    4 for history and cultural since this is a confusing selection and I already think the nomination isn't bad. 4 isn't punishing a nomination for an already confusing review choice.

    5 for visually unique since this is "can people find it" rather than "is this place special". Don't confuse this point. You can definitely find a park even though it might not be "unique" in the sense people think.

    5 for safety most times since it would have to be in a strange place not to be reached. Remember some areas (like the rural area where I live) won't have sidewalks so one can't force a "there must be a sidewalk to be safe" view on everything.

    5, 3, or 1 for Location: 5 (it's where it is), 3 (probably there but can't see it), 1(cannot verify it but not about to give it a 1 rating overall right at the start)

    Remember that access doesn't necessarily mean everyday access to the general public. Some good nominations will not have access to the general public yet still be accessible. Think something like a statue of Henry Ford at an auto manufacturer.

    Bad nominations:

    1* and goodbye for super obvious bad nominations

    1* for nominations that look to be truly cheating the location (as in not really there rather than a few metres away or even 50 metres away if there aren't any buildings around to make me feel the person did it on purpose. Sometimes pin placement is hard in wooded locations).

    The middling ones can be difficult but aren't the bulk of nominations. You can usually figure out if it's good enough or not. If I'm honestly unsure I'll skip or let a nomination time out.


    People may disagree with my way and I may change my system again but I feel it's fair. If I think something about it isn't as good I'll rate it lower in that section. I don't race through nominations. I do my best to find them using street view or birds eye rather than just thinking "**** it" the pin isn't in Street view so 1*. I give them every chance but can recognize bad points. Just wish we had a few more "skips" so I could comfortably continue rather than timing out reviews I'm not sure of.

  • KotozaY-INGKotozaY-ING Posts: 34 ✭✭✭

    It would be very interesting to learn about this theory, because even though Niantic has given explanations on this subject, people keep getting rejections for good candidates, seeing the reasons as lack of cultural/historical relevance or lack of visual uniqueness.

  • Hosette-INGHosette-ING Posts: 3,469 ✭✭✭✭✭

    @KotozaY-ING I've given my theory several times.

    I think that the thing that generates rejection reasons is very bad at its job. I don't think the reasons it's giving are actually rejection reasons, at least not entirely... I think it's giving some combination of "hard" rejection reasons and "here are some categories where reviewers gave it a low rating."

    Something interesting: If you look at the possible rejection reasons, most of them could theoretically come from either reviewers giving a 1* rating for "Should this be a Wayspot?" or from getting a low rating in a category. It's impossible to know, for example, whether the rejection reason for pedestrian access comes from a 1*/Location/Pedestrian access rejection, from a low star rating in the safe pedestrian access category, or possibly from both. The two things that drive people crazy, historical/cultural relevance and visual uniqueness, don't have an analog in the 1* rejection reasons.

    Here's a puzzle of sorts: Imagine a candidate that got 2* ratings from every single reviewer for every single field. We know from an earlier AMA that this hypothetical voting pattern would result in a rejection. The code that generates rejection reasons wouldn't have anything to use from the 1* hard rejections, so what would it do? My guess is that it would still generate rejection reasons only they would be based on the star fields rather than "Should this be...". It's impossible to guess what order they would come in but I'd guess that it's either based on some predefined priority or in the order that they're defined in the code. I wouldn't be surprised to see historical/cultural show up in that case even though the real reason for rejection was "the overall star ratings fell below the acceptance threshold."

    I'm the one that asked that question in the AMA, and I did it in the hopes that this persistent rumor would die once and for all.

  • VladDraco-PGOVladDraco-PGO Posts: 560 ✭✭✭✭

    Hi and welcome !

    To answer your questions :

    1 : Yes, each nomination goes for evaluation by multiple reviewers. It takes times for a submission to be accepted or rejected.

    (So your numbers and ratio +1000 = 108A/111R seems totally normals)

    2 : Each of us has a different way of translating 1,2,3,4,5* in words. Mine is 1*=NO !, 2* is No but I cannot really be sure why, 3* is Ok but you didn't really convince me, 4* is Yes, ok and 5* is Great, Absolutely !

    3 : It's complicated. @Hosette-ING is explaining this better than I can.

    4 : By promotion you mean Poor,Fair,Good and Great Rating ?

    It evolve depending how much percent of the final decisions on submissions are the same as what decision you choose on them. In short Are you on line with other reviewers.

    Your rating give you the impact you have on decisions : Poor = you have no impact, your decisions are just tests to help you evolve to Fair. Fair = low impact, Good = normal impact, Great = Boosted impact.

  • AddyLaddy-PGOAddyLaddy-PGO Posts: 2 ✭✭
    edited January 2022

    Thanks for all the tips everyone. I also found more guidance in the Help section of the Wayfarer site.

    I think my confusion stems from the disconnect in time between me reviewing a submission and the totals and Wayfarer Rating being updated.

    Also, I think I misunderstood the Agreements totals, I assumed that "Nominations Rejected" were nominations that I had accepted but the community had rejected. My understanding is now that those totals: Accepted/Rejected/Duplicate are reviews where my decision has matched the consensus. Am I correct ?

    My rating is still "Good" and I'm now at 2162 with 217/253/37. So ~500 agreements. Can I find out how many of my reviews are still pending consensus ?

    -David

  • Hosette-INGHosette-ING Posts: 3,469 ✭✭✭✭✭

    @AddyLaddy-PGO Yes, you are exactly correct... the Accepted/Rejected/Duplicate numbers are the ones where the review has reached a decision and you agreed with the consensus.

    There is no way to find out either of the numbers that you would need to know how many are still pending-- you would either need to know the number pending or the number of times you disagreed, and Niantic doesn't give us access to either of those numbers.

Sign In or Register to comment.