Good reviewing lowers rating?

I want to start this discussion, because it is something I struggle with, and want to hear other's perspective.

Here's the scenario: you are reviewing a nomination that is almost perfect, except for a minor detail. You really like the object that is nominated. But you know that the majority of the reviewers will accept (because they say "don't overthink", they are idiots, or something else). That causes the following dilemma:

- rewarded with rating for bad reviews

- be the noble moral crusader, give a fair review and face the consequences

Examples:

- a marvelous statue that is on a roundabout

- a beautiful historically valuable stone carving on the façade of a historic building in the historic center of a city, but it is still inhabited, i.e. private residence

- a niche in a façade with a Mary statue in it (the niche is pr.p., but the statue is not connected to the façade, and stands on the property line), in Belgium (in practice, every Mary gets approved)

Rejecting these would be good reviewing, according to the rules. What do you think? Go with your guts, or review with the iron fist?

Comments

  • gorgi1978-PGOgorgi1978-PGO Posts: 283 ✭✭✭

    I was just reviewing according the criteria and guidelines as much as possible.. As I understood, there will be some honeypots which will influence your rating as well.

    And the more you review, the less impact a (valid) different opinion has.

    Nonetheless: sometimes what you consider as invalid could be valid. For instance, trail markers and private property are examples that are often misunderstood.

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭

    I agree, although... When encountering a nomination for a historic farm in rural Europe, can you differentiate if it is uninhabited heritage, or a single family's farm/residence? I can, but I find it very unlikely for people to know their way around land registries.

  • gorgi1978-PGOgorgi1978-PGO Posts: 283 ✭✭✭

    Partly it will be the job of the nominator as well, to 'sell' it as much as possble. For instance, by including references. If it's a farm, you see a car, on google maps you don't see that's its a business, and nothing special is described about the farm, it will probably be considered be PRP.

    As a reviewer, I'm verifying the data, being interested in the nomination as well. I don't fill in the blankets and don't search for the missing information on what makes the nomination interesting. The latter is the job for the nominator.

  • saarstahl-INGsaarstahl-ING Posts: 183 ✭✭✭

    Reject if it does not meet criteria. You actually can't know whether the majority of the reviewers will accept.

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭

    Yes, you actually can. When both reviewing and nominating enough waypoints, you'll start to see patterns, because of what gets approved and what doesn't.

    For example, most reviewers have no idea about architecture, and no matter how famous. But I get that. When you can count all the world famous architects you know on one finger, any other architect will sound very obscure.

  • rufoushumming-PGOrufoushumming-PGO Posts: 1,373 ✭✭✭✭✭

    Good question. Follow the rules. You will not be judged harshly for it. And your ratings will not fall. If you have any fear always put a comment why you rejecting.

    This religious icon is part of the facade of a private residence.

    This statue is on a round about without safe pedestrian access

    Does not matter who by, what heritage, who connected to it - Just because a nomination is eligible (exercise, explore, socialise) does NOT equal acceptance

    End of story. Back yourself but to be safe. Always provide the reason why.

    As for patterns. Of course there are patterns. Humans are sheep we mimic what we see. If that was accepted then this must be - now I don't have to think. Then there are voting cabals who nominate and vote to get their thing through and work every channel they can to make you accept their views. Then there are auto bots. Then there are those that only want to get their reward. Then there are those who want to nominate the most. Or those who just enjoy gaming the system. They all have an impact and will create patterns. And as I said we are sheep so we will follow what we see.

    And Niantics onboarding and ongoing education process leaves a lot to be desired so people do not learn pro-actively.

    Ignore them. Do the right thing. As I said - Does not matter who by, what heritage, who connected to it - Just because a nomination is eligible (exercise, explore, socialise) does NOT equal acceptance.. That is the end of the story. Back yourself but to be safe. Always provide the reason why.

    Your rating will not fall if you do the right thing.

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭

    The right thing? That sounds like the morally correct thing to do. But I don't want that anymore. I feel like experience the cognitive dissonance caused by caring to not caring, caused by disappointment: Niantic doesn't show they share my values on morality, honesty, transparency and fairness. If a system would work, and be fair, transparency wouldn't be an issue.

    I also disagree that good reviewing doesn't cause rating to suffer. I suspect that, now that your mention autobots, every now and then for a few days or so, everything gets accepted; even my "giving it a shot" kind of nominations. So that might play a role.

    For a while I did just that, explaining my probably controversial judgement. That made no difference.

    Maybe it's time for me to stop reviewing altogether.

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭

    Reading this back, I realize I let my disappointment get the better of me.

    Seeing the featured waystop at the moment gave me a moment of clarity: we need more good reviewers who don't care about rating, act like first-line gatekeepers, because we are passionate about wayfaring, and don't want our map to become cluttered with coal.

    Once said, a wise creature probably:

    Frustration leads to disappointment, disappointment leads to cynicism, cynicism leads to having a bad time.

    If you would excuse me, I've got a featured wayspot to report😎

  • Studogvetmed-PGOStudogvetmed-PGO Posts: 10 ✭✭

    Total Nominations reviewed 4582, 1798 accepted, 948 Rejected, 145 Duplicated.

    Once my Rating settled in Great, it has never fallen below Great.

    If you review enough, and follow the criteria. It all comes out in the wash.

    I'm a fair weather reviewer as I like to say. I do my most reviewing when I have actually something in the queue, I review less when I don't have anything in queue or in voting. I help with the backlog when I add to is my adage. I can't justify to make this my second day job.

    Remember that even in the criteria there are shades of grey, but don't fall back on obvious rejection criteria: PRP, No safe acess, K-12, etc. Remember that location can lead to shades of grey on 'safe pedestrian access', but even that can be difficult.

    I have seen multiple stops approved that I have 1* rejected, it has not affected my rating.

  • Elijustrying-INGElijustrying-ING Posts: 5,316 Ambassador

    @MennoLente-PGO

    The process you went through was actually something we all should do more of. Everything we do in wayfarer is about judgement. Not something hard and clean, but soft and murky (hope that translates well 🤔)

    It is easy to become complacent, comfortable in how we are reviewing. Changes in what we are seeing in the nomination flow changes often very slowly. And then we see new wayspots appear and raise doubts.

    We need to stop and reflect every now and then, talk it through, challenge our thinking and perhaps reset. This is difficult to do when so many of us are quite isolated. Chatting to someone challenging them is a natural way.

    You have ended up at the right place.

  • KyleBaps-PGOKyleBaps-PGO Posts: 127 ✭✭✭

    I get the frustration here to be fair, it's actually quite striking to me as I saw something misplaced quite clearly had made it through today and was shocked by how anyone could have accepted that without moving the pin correctly. I often see a fake, a misplaced pin, something on private property or a very generic sign and reject/correct the pin on what I believe to be the correct and fair assessment, only to later see another fake or misplaced pin appear in the same area and find the previous one had been accepted, which is sometimes frustrating when we know there is abuse going on there.

    I think so long as we review to the best of our ability, what gets through shouldn't deter us from doing our best with reviewing. It's best not focusing on what others might vote something but what we believe to be fair and correct, to the guidelines and criteria's we have been provided. I take it upon myself to try and leave a comment on more niche acceptances/rejections/low rated/misplaced nominations about my choices so Niantic are aware of why I voted or moved pins in appeals or potential future investigations if fakes are in the area or the area is being abused.

  • DukeOfBellaire-PGODukeOfBellaire-PGO Posts: 219 ✭✭✭

    Good rejections increases your rating.

    Bad acceptance decreases your rating.

  • thenamelesskath-PGOthenamelesskath-PGO Posts: 311 ✭✭✭

    Just to add to that, whenever we get to this point of questioning our understanding, we should first be returning to the criteria pages (both sets, until they bother merging them), AMAs, release notes, criteria challenge(s), etc, to reconsider the official information through our shifting lenses (with an open mind); because while forum debate can be great for more points of consideration, it can also muddy the waters or increase confusion, when you're already feeling completely overwhelmed by the details.

    I personally like to go right back to basics if I find myself getting caught up in confusion over nuanced details and being overwhelmed by them (these graphics are fantastic for this and should be one of the most forward-facing things for Wayfarer onboarding, IMO):


    My understanding of the criteria has evolved quite significantly in some areas and I've realised a few times that some of my understanding was based on assumptions, criteria details that were changed/removed, or statements that weren't actually made by Niantic themselves 🤦‍♀️

    I've now taken to screenshotting Niantic clarifications, from their various scattered locations, so that I can refer back and keep track of them more easily (since they don't bother taking these opportunities to clear up confusion on the criteria pages). I often use them in submission supporting to try to help spread any newer context to those who don't engage with the forum, as well.

    It's easy to develop a fairly rigid dichotomous mindset (especially as we get older and brain plasticity reduces lol), but most of the Wayfarer criteria is just not set up that way. Usually, either the criteria itself or its application is open to interpretation (even where many Wayfarers still refuse to see the potential for any interpretation but their own) 🤷‍♀️

    Kinda funny to be the one to point it out, given a pathological inclination to dichotomous reasoning 😅 Maybe it comes with experience, education, insight, and having to actively manage it 🤔🤷‍♀️


    Having said all that, if Niantic expects us to keep focus of what's "really important to us as a community of Explorers" and move away from "discriminating against nominations categorically, and asking us to think more holistically about what we consider as interesting and important to our local communities", then they absolutely need to do better, themselves! They need to be much more considerate and succinct with their wording, to ensure that the various parts of the criteria are not so contradictory that most Wayfarers will struggle to make sense of it all, holistically (it really is just too much to expect, evidently) 😬🤦‍♀️

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭

    This is exactly why I came here. I needed that encouragement, and to be reminded that I could be right, but that doesn't change anything for me. But my passion is helping others (if it benefits me too, that's extra. I like extras), it's in my nature.

    That is why I'm a wayfarer.

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭

    I fail to see how that addresses my specific points of frustration and critique, but I agree, except on the part of the brain plasticity.

    To me, that explanation sounds like somewhat oversimplified, with to many exceptions. While it is true that the older you become, the more things you learn that you assume "factual". Challenging existing beliefs is always harder than excepting new ones, therefore creating the illusion of old brains learning slow. That's why there are old people who are still using landlines, while others seem like their brains are immortal (one acquaintance of mine is almost 80yo, who successfully wrote an app to use his smartphone to control his electric wheelchair, without any prior knowledge🤯).

    I think that that explanation better fits perceived reality, but as always, feel free to challenge my views. I like the feeling of having learned something.☺️

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭

    Yeah f* that rating, they can shove those agreement badges where the pr.p. rejections go! We rate fair, honest and according to the guidelines, concensus, and good taste (if applicable), holistically! 💪🏻

  • thenamelesskath-PGOthenamelesskath-PGO Posts: 311 ✭✭✭
    edited October 2023

    I was only making a point in addition to the one already made by Elijustrying, to encourage better practice in updating our understanding of criteria, to reduce confusion and increase quality of the learning process; and to point out the difficulties imposed by the current system on this. As such, it wasn't intended to be a comprehensive explanation of all the issues with Wayfarer, but point taken, and I apologise if you feel my comment lacked merit or was off topic 😥

    To reframe in relation to your concerns, my point really was just that, when asking these kind of questions, we should first consider if our own understanding of the criteria is up-to-date and the fairest representation. To do so, we should rely on Niantic materials first (as the primary source of fact, such as it is), and then consider other viewpoints for further context/clarity, if needed. It could be that your iron fist understanding of the rules isn't as accurate as you thought, and your gut feeling was a better holistic representation; especially where regional differences exist 🤷‍♀️

    However, if after reviewing the current resources, you still believe your understanding and application of the criteria is the best it could be, then you should review only according to that and not worry about your rating. You will be more harshly punished in the long-term if you're willfully reviewing incorrectly.

    If you believe that others are very far off the mark in their submitting/reviewing behaviour, then you can either report the submissions through help chat, or wait until/if they go live and then report them for removal. Doing so should help Niantic eventually catch up with, and hopefully educate, people doing the wrong thing; which should affect change in local reviewing behaviours in the longer term.

    I think you'll always struggle if you entangle your reviewing with ideas of morality, because there isn't necessarily black and white answers to reviewing choices involving personal judgements, and people who have different views don't necessarily hold them for nefarious reasons 😬

    While there are certainly those who intentionally do the wrong thing, I think they would be the minority. As I keep trying to point out to people, there just isn't necessarily a right or wrong answer to how we interpret much of the vaguely worded criteria or their application, so placing moral significance on these grey area choices is pretty unreasonable.


    As far as the rest of the points you raised, I think maybe you misunderstood my offhanded mention of brain plasticity, and I'm sorry if I offended you with it. While it will be a factor, the comment is probably better represented as a self-deprecating joke; but that's also all wrapped up with "baby brain", which is a whole other thing 😅

    It's a simple fact that "as we grow older, plasticity decreases to stabilize what we have already learned". It's not to say that we can't learn new things or are completely unintelligent or unadaptable, just that there is a physiological reason that humans get set in their ways and it is something we have to work against; if we so choose. In many instances, such as your landline example, if it works for people then they're unlikely to want to learn something new (or even with things like music, people tend to get less open to newer stuff that sounds different).

    So from a Wayfarer perspective, there's a variety of reasons that people may be set in outdated ideas of the criteria, and it can be just one of many factors contributing to unideal reviewing. This can affect ratings, whether you're the one who hasn't yet adapted your reviewing, or when the firm held, but not necessarily correct, views are more common (and I'm not making any judgements on that) 🤷‍♀️

  • Johnsonsine-INGJohnsonsine-ING Posts: 405 ✭✭✭✭

    I think the examples you give have nothing to do with being 'a noble reviewer', honestly. There is a reason we shouldn't accept POIs on private property or in roundabouts. It's not even complicated, it makes perfect sense. These Niantic games are fun and all, but not worth getting hit by a car over (in a roundabout) or invading someone's personal space over (private residence needs to be respected!). These two specific rejection criteria are just so easy to understand and they make pefect sense.

    I think it's wrong to call people "bad reviewers" because they respect and understand the criteria. If what you're suggesting is an example where the roundabout is very small and can be crossed (?) or maybe the good old "the pokestop can be reached from the nearby sidewalk" (??) then maybe it's you who need to re-browse the criteria?

    There will obviously be grey-zone areas in Wayfarer, that is mentioned too in the guidelines. I think what a lot of people do is: When in doubt, be very literal in the reviewing. Example: The Mary statue is juuuuuuust on the property line - to be safe, I better reject it. And not only to accomodate what others may vote, but also to respect the actual thought behind that specific rejection criteria. Respect people's privacy.

    That said, I know there are many examples of local communities all over the world that just kinda silently agree to ignore a given criteria. In Denmark, it's the 40 meter rule in playgrounds. I will even say it's more uncommon to see a playground that actually has 40 meter to the nearest private single-family home than one that follows that guideline. So... Yeah.

  • thenamelesskath-PGOthenamelesskath-PGO Posts: 311 ✭✭✭

    Hey @Johnsonsine-ING, it should be pointed out that there is actually no "40m rule", the only mention of "within 40m of PRP" is that we should review these very carefully to ensure they are not, in fact, on the PRP. POIs within 40m of PRP that can be confirmed not to actually be on the PRP are perfectly acceptable 😉👌

  • Johnsonsine-INGJohnsonsine-ING Posts: 405 ✭✭✭✭

    They removed the rule? Oh, that's fine. Nobody followed it anyway... I can totally understand though why home owners would NOT want a gym on the playground right next to their house, h.ll, I can understand how ANYONE - even the lowlife people like myself who have to RENT lol - would not like screaming and shouting about shinies right next to the fence. Been there, done that. Raid Days were HARD. Tried getting the gym removed but Niantic don't gaf about us. 😂💀

    It's hard to find pretty much anything in this chaotic forum, it's like a 9-year old set it up. I tried searching for the comment made by some Niantic person about the 40 m rule, but gave up after ten minutes or so. I've seen it though, but have no idea if it still applies.

    And again, nobody respected it anyway. Including Niantic.

    So it's fiiiine.

  • thenamelesskath-PGOthenamelesskath-PGO Posts: 311 ✭✭✭

    Yeah, there is probably remnant discussions floating around, but it hasn't been worded that way for years 👍

    TBF playgrounds are places where rambunctious and noisy play is expected and encouraged, so the specifics of the play are quite irrelevant and complaints would generally be considered unreasonable 🤷‍♀️

    Here's a thread that might prove illuminating 😉


  • Johnsonsine-INGJohnsonsine-ING Posts: 405 ✭✭✭✭

    Playgrounds are noisy indeed, and that's exactly why it makes perfect sense not to add anyMORE noise to the location. 😉

    Again, PoGO players can be LOUD.

  • MennoLente-PGOMennoLente-PGO Posts: 83 ✭✭✭


    First, no need to apologize! If anyone, it should be me to apologize, with my tendency to black/white thinking. Perhaps it might be even a cultural thing, as the Dutch have a reputation for having strong opinions about everything.🫣

    Thanks to your lengthy (please don't apologize, I love lengthy!) clarification, that explains my confusion. It reminded me of how much I value being able to change your opinion when presented with contradicting evidence, but you still need to recognize it as evidence against your own flawed opinion.

    But you may be right. And I like to have my views and opinions challenged, because I see it as an opportunity to learn.

    I thought long and hard about every one of your comments, and came to (what I think is) an interesting conclusion, that is completely missing anyone's point, including my own! 🤭

    So, how would I design a better system to rate nominations? Well, I use LLMs on a daily basis. As autistic person, I thrive with clarity, and feel held back when vagueness is involved, that's where LLMs like GPT and Bard come in handy (and can even be essential to me, except Bing. Bing judges me for saying inappropriate things, even when I didn't mean to). So I wrote a prompt that analyses the photo, looks if it resembles or represents the description, added as much rules and guidelines as needed, gave it a scoring system, and it rates as well as you'd expect (except that I couldn't get a few features to work without plugins, like access to land registry database, zones and objects from OSM, and was forced to use bard for photo recognition, since I only have access to free version). But it's still good enough as a substitute for a reviewer.

    I came to the conclusion that guidelines are too easily bent for practical use by persuading use of words (and sensitive to prompt injection), and rules can't cover everything. Also, don't follow too many rules, both the human, as well as the AI get confused, and doesn't improve results.

    Acceptance guidelines combined with rejection rules proved to give a realistic simulation, equally unpredictable as it is now.

    What worked best, was acceptance guidelines, combined with category specific rules, combined with rejection rules.

    The AI judged both existing as well as non existing the same as I did. That was not intentional, because I don't see myself or the rejection criteria as flawless, but it's a hopeful sign.

    So I'm going to start with the human-assisted AI reviewing, and I am curious to see which way my rating will go.

    Note: of course, fully automated would be a breach of ToS. Assistance from AI with me making the final decision isn't mentioned in the ToS.

  • rufoushumming-PGOrufoushumming-PGO Posts: 1,373 ✭✭✭✭✭

    I still do not understand what the consequences are for reviewing correctly.

    When I say do the right thing. Not a moral call. Just follow the Niantic Criteria. Niantic Wayfarer (nianticlabs.com) And if you reject something explain why you rejected it. Want to add a note because you gave a low rating give the note. Moved the POI explain why.

    I reject constantly. I accept constantly.I duplicate as appropriate. I put the comments in.

    Now of course. The criteria and often the Criteria Challenge updates are cloaked in ambiguity. And as we all nominate or review for differing reasons. We are driven to do things in particular ways based on how we feel - we have automatic behaviours based on past learning and in built behaviours. We might want to help wayfaring grow. We may just like to **** systems up. We may be part of a voting, nominating ring. We may just want to get a medal/reward. We react because we see a DB full of accepted POIs and we be sheep. Etc etc etc

    The ambiguity is good as it enables nuances to cater for differing cultures, places etc. As a rural town varies so much to the city centre. But is flawed as human nature seems to prevent many people from stepping outside of their own why am I doing this reviewing lens to accommodate those variances.

    The nice thing is I can come here to the forums and get admonished or congratulated in equal measures. But I learn when I come here.

    I applaud wanting to do better. And to create a better DB of nominations. And if AI/ML tools can help you do your nominations better. Then why not. After all Niantic uses AI to judge us.

    It sounds like you are building quite a powerful tool there to assist you. Be great to hear how you getting on with it. Probably be very useful in the next review challenge :-)

Sign In or Register to comment.