An easy solution to our reviewing woes

Priice-PGOPriice-PGO Posts: 12 ✭✭
edited November 2020 in General Discussion

If you've reviewed at least 100 submissions, you'll come across some really terrible stuff. We understand the issue -- A large influx of people were given the keys to a complicated system with little to no understanding of how it works. While the Wayfarer team is working on ways to improve the situation, we need to look at the cold hard facts: The reviewing queue is awful. More and more I'm stumbling upon submissions in the middle of nowhere that don't even need to exist. Even a reviewer with little understanding of the system would know that these submissions are dumb and a waste of time. It comes down to a lack of understanding.

We need these people to take a test before they can submit. The release of Nominations to Pokémon GO players solved the queue for a while, but now we're being bombarded with absolute trash a year later. Ingress players have the potential to submit junk too, but the ratio of Ingress to Pokémon GO players with the ability to submit is so diluted it's not even worth mentioning.

The Solution I'm proposing may have already been stated, but I'm going to reiterate in an articulate manner: There is now a reviewer medal in Pokémon GO. Medal achievement is not anything new to Niantic's games; Why is this medal not tied to the ability to submit? Gold requires 1000 agreements. 1000. By the time I achieved this number, I fully understood how submitting worked. There's a reviewer score for a reason; if people give agreements to trash, their rating turns to trash. There is now a solid reviewer base in the PoGO community; let's increase that number by forcing submitters to do reviews first.

PoGO players now receive the ability to review at level 38. That's halfway to level 40. That's PLENTY of time to get reviews done before they submit. And it forces our current trash-submitters to take another look at what they're doing wrong.

TLDR; Tie the newly implemented Wayfarer medal in PoGO to player accounts. If it isn't gold, they can't submit. That's 1000 reviewer agreements before they're allowed to submit anything.

Post edited by Priice-PGO on

Comments

  • LukeAllStars-INGLukeAllStars-ING Posts: 4,625 Ambassador
    edited November 2020

    I would make a stop for submitting until everyone has at least 100 agreements. After that, old submissions are finally reviewed and the players know a bit better how/what to submit.


    I know this is a bit controversial, so I am interested in your opinion in this^^

  • JohnnyAlphaCZ-INGJohnnyAlphaCZ-ING Posts: 235 ✭✭✭

    I'm okay with this as long as there is a similar rule implemented for Ingress players. PoGoists are not solely responsible for the dross in the review queue. A lot of rubbish was submitted (and some approved) long before the PoGo players were let in.

  • Priice-PGOPriice-PGO Posts: 12 ✭✭
    edited November 2020

    The Recon medal requires 750 agreements for Silver. I think that's a fair trade. The Gold medal (2500 agreements) might be a bit much for attaining the ability to submit.

    I don't think 100 is enough, personally; but it would be a start. I think we'd see an influx of trash agreements if it were just 100 reviews.

  • JohnnyAlphaCZ-INGJohnnyAlphaCZ-ING Posts: 235 ✭✭✭

    I think 100 might be enough. I get what you're saying about the influx of trash agreements, but the same time we don't want to make the number so high that people just don't bother. Niantic need to show constant database growth to attract clients, but they also need to show database accuracy. It's a fine line to tread.

  • Priice-PGOPriice-PGO Posts: 12 ✭✭
    edited November 2020

    Grandfather in current submitters to a Silver medal, but level 38 & 39 players need the Gold from this point onward. Or, maybe an increase in submissions with a higher reviewer medal? 1 submission per 2 weeks versus the 7 they get now until they get the Gold medal.

  • Ultimo6419-PGOUltimo6419-PGO Posts: 88 ✭✭✭

    Just yesterday, i suffered while reviewing plenty of 1* nominations. I think we need several improvements.

    Pokemon menu says: "submit pokestop" and supporting text is encouraging to "create pokestops". Players get a false impression about what they are doing. Niantic is planning to improve the communication, but nothing has changed until now.

    Pokemon players/submitters do not see all portals unless they open an additional ingress account. As existing portals outside PoGo do not show within the nomination procedure, we face several duplicates.

    Education / Testing. I think it is unfair to close the door forever to anyone who didn't pass the first two review tests and leave all other doors open always. Testing should support reviewers and nominstors. A test every half year/ no. of submissions with answer-feedback would be great. If reviewers/nominators can take the test as often as they want but can only continue if 80% of answers are correct, that would be a benefit for everyone.

    It won't stop fakes though.

  • rodensteiner-INGrodensteiner-ING Posts: 1,686 ✭✭✭✭✭

    yes, the main problem is still, some PGO people do fail both test on how to review, but will still be able to submit with 40.

    that is like giving someone the drivers license if he failed the paper test, but got thru the practical test. In the end, this will lead to car crashs.

    Same Car Crashs we sometimes have to review on Wayfarer.

  • Damastaglen-PGODamastaglen-PGO Posts: 124 ✭✭✭

    I think the players of PoGo and Ingress (and maybe HPWU at some point) should at least have to do the Wayfairer tests before they submit because whereas something may be cool and perfectly acceptable as a POI to other companies, they need to understand what is acceptable to Niantic (such as the minimum distance requirements between playgrounds and houses).

    I disagree with saying they need to do reviews to get nominations because it doesn’t really make sense. The thing is, if they are ‘qualified’ enough to say if somebody else’s nomination is legit then they’ll be qualified to say if their own is.

    Do I see rubbish nominations? Sure - I 1* and move on. But even if I do 1* for whatever reason, if it’s cool and nearby I may still go check it out.

    Have these come from Ingress players as well? - well some have references portals so I assume so.

  • donandlan-PGOdonandlan-PGO Posts: 201 ✭✭✭

    Doing x number of agreements/reviews before being allowed to submit sounds good in theory. However, I don't expect reviews to make better submitters when it is pretty clear that a large portion of the reviewers don't have a solid grasp of the rules themselves. We've all seen bad submissions that we correctly 1*, or even report for abuse, that somehow get through the process and made live.

    Niantic needs to overhaul the review system. As of right now, I have a great rating, but as we all know, there are rules, and then there are how the groups vote, and the two are not always the same.

    There is zero usable feedback on how good my reviews are. Are the 30% or so non agreements because I'm too harsh, not critical enough, are they all fishing points/pier/boat launch disagreements? Who knows... If I knew, I could review more effectively.

    Niantic has announced some changes coming down the line in the reviews, and clearer examples, simplification and explanation of the individual rubric scores on the review page, and better feedback about my reviews will certainly make things better.

    They also need a singular, concise set of guidelines so we're not constantly searching discussion boards for guideline clarifications.

    Until all these come to fruition, I don't see how forcing players to begrudgingly participate in the tedious, haphazard review process is going to magically make them better submitters.

  • DukeOfBellaire-PGODukeOfBellaire-PGO Posts: 219 ✭✭✭

    In brief: complete 1,000 agreed reviews before one can make the first nomination?

    Perhaps just 50 agreed reviews will be enough. Remember that many of your reviews will remain pending, unknown, or disagreed.

    But without feedback upon disagreed reviews, there is no way for the reviewer to understand why their review did not collaborate with a group of reviewers and Niantic. A common rookie mistake might be nominating and approving Starbucks.

  • Hosette-INGHosette-ING Posts: 3,425 ✭✭✭✭✭

    One thing that has to be done with every rule change is to see how it changes the incentives for player behavior, and also to look for ways to game the system.

    I suspect what would happen is that a lot of people would start rating everything 1* until they got to the threshold number of agreements. If enough people did this, and we've seen it happen before, legit reviewers would have their agreement rate drop and the players downvoting everything would end up with a high rating.

  • 52cucumbers-ING52cucumbers-ING Posts: 225 ✭✭✭✭

    More and more I'm stumbling upon submissions in the middle of nowhere that don't even need to exist. Even a reviewer with little understanding of the system would know that these submissions are dumb and a waste of time.

    I'm sorry, but what? Just because you don't see any possible use for a waystop "in the middle of nowhere" doesn't mean that it don't need to exist. Review based on the criteria and not how remote the POI is.

    I agree with everything else in your post.

  • AgentCubebot-INGAgentCubebot-ING Posts: 25 ✭✭✭

    I agree that everyone should pass a test before submitting. I don't know what people are thinking when they submit dog poop-bag dispensers, stop signs, and other generic items.

    Saying they need to complete XX agreed reviews before one can make the first nomination does not help if they don't read and understand the guidelines or know what ones that they have reviewed are actually are good. If they saw 100 submissions for apartment complex signs, then they will go and nominate every apartment complex in their area.

    The Pokemon menu does says more than just: "submit pokestop" and "create pokestops" but somehow people click next on these and proceed to submit a manhole cover.


  • Priice-PGOPriice-PGO Posts: 12 ✭✭
    edited November 2020

    Proximity isn't the issue, it's that a staggering portion from these areas are "Ned mor pokstop" and it's a picture of their garbage can. I actually seek out poi in desolate areas to help out the locals, and I submit on remote hiking trails. The point is that people from these areas have no idea there's a community for reviewing and think "The Nintacs" does reviewing for them. They submit trash and get mad when their Nomination never shows up in-game, so they submit it again.

    Some of these people are trolls, but others are just uneducated.

    Post edited by Priice-PGO on
Sign In or Register to comment.