Make Appeals Earnable, Like Upgrades
ScrawnyMeowth1-PGO
Posts: 191 ✭✭✭✭
Problem #1: Niantic is way behind on reviewing appeals. They seem to have made progress in recent weeks (my November appeal is now on voting) but there's still an 8-month backlog.
Problem #2: One appeal every 30 days is still not enough for people who submit a lot. Just last week I went on vacation and submitted 30+ nominations. At least half a dozen so far have come back rejected for bogus reasons, and I can't resubmit now that I'm back home.
Problem #3: Many people are appealing ineligible nominations. I've seen instances where players (usually new to Wayfarer) submit something clearly ineligible, get it rejected, but then appeal it "because why not," without stopping to ask if it's actually eligible. This is adding more unnecessary appeals for Niantic to review.
Proposed solution: Allow players to earn appeals by getting Agreements instead, just like Upgrades. Because the people who review a lot are somewhat in the minority of Wayfinders, I believe this will cut down on the overall amount of appeals. At the same time, it will allow the people who submit a lot to earn more appeals for their nominations, since the people who submit a lot usually also review a lot. Finally, this will ensure that newer Wayfinders will have to do some reviews before they can use an appeal, which will hopefully help them learn more of the criteria before they submit. In my experience, doing reviews is one of the best ways to learn the criteria. And of course, putting appeals behind Agreements will encourage more people to review.
I'm not going to suggest a specific amount of Agreements that should be required for an appeal, since that goes into complex game balance issues. However, I do believe that my general solution, if implemented by Niantic, will help reduce some of the problems associated with appeals right now.
If anybody disagrees with my suggestion, I'm happy to hear opposing arguments.
Comments
One option would be that you could use your regular upgrades for appealing. this would make you think twice about appealing, because you would waste your upgrade if the nomination is bad. i guess this would lead to less overall appeals in the queue.
@NianThib @NianticAaron @NianticTintino-ING
I would like to see both submissions and appeals be earned. Grant everyone something like one submission per week and others can be earned in the same way that upgrades are. Maybe grant each person one appeal per quarter and they can earn more by reviewing.
If a nomination was upgraded, its appeal should automatically be upgraded.
To upgrade an appeal (on a nomination not upgraded) should take fewer agreements than the 100 for an upgrade. Maybe ten. With a free appeal every 60 or 90 days.
BUT, all this is shortage thinking. Niantic should fix their systems and have an abundance thinking. Wouldn't it be marvelous to have turnaround like Google Maps - a couple days! Instead of Niantic's years.
I would like to see some sort of a point system where agreements earn you points. Then you could spend those points on various things, and not just upgrades and appeals. You could spend points to get more submissions, send things directly to Niantic review (for things that have 0 chance with local revs), upgrade edits and photos (when those are added to submissions), maybe even allow us to update image or coordinates for a fee.
The experience a wayfinder has is already very different depending on where you live.
Live somewhere that has priority then Upgrades and appeals are not needed. You submit something and within a week it has been resolved. No need for an upgrade. If it’s rejected then no problem, redraft and resubmit and a week later you get resolution number 2. Still rejected? Then just repeat. All the time your repeat submissions are getting prioritised. No need to review, no need for upgrades or appeals.
Live somewhere that is very slow (without intervention from Wayfarer these were at in excess of 2 years, now still around 16 months). If you don’t do the reviewing for upgrades then you are going to have to be very patient. You wait all that time and it’s rejected. Although appeals are slow it’s still a fair bet over a resubmission. You probably have to resubmit as a side bet. So doing reviewing for this group becomes essential.
I too have been holiday. The sheer joy of submitting and getting a result in days adds to that Wayfarer holiday experience. Back to reality.
I would much rather see parity of experience, than making matters worse by locking access to these behind reviewing.
Thank you for the suggestion. I'll bring it to the attention of our concerned team.
Regarding backlog, we have taken steps to fasttrack the process and reduce the overall backlog. You should see significant progress in weeks to come.
Maybe there would not be so many appeals if there not so many shoddy rejections.
I feel Niantic is stuck between a rock and hard place. Which to do first. Priority list of other work. Business fundamentals at play. Staff V amount of work. How to drive revenue? How to reduce costs? They don't do this for free. So hey... what has more value.......
My view:- is you need to improve educating people when you onboard them to nominating and reviewing and have this as an ongoing process and not just yearly.
With loads of nudging taking place (nudging is a psych technique to get help steer behaviour)
Continue educating every year (quizzes?). Honey pot traps. Remove PGO medal for reaching X number of and so. So people slow down. Which is important when Niantic is competing against a bazillion things wanting their users eyeball time. A lot of reviewers if they to do an inch of work to verify/validate they reject. And rejecting is quicker than approving. And if you want to get to that medal quickly then hey.... And if you in a cabal in a local area you move as quickly as you can to get to the nomination your cabal wants you to approve. And I have not got near people who just like to mess with the system for fun
And far be it for me to say this . But doing reviews to get more appeals will do little to zero in getting improvements in people understanding criteria or becoming better. See previous paragraph.
For any of us (myself included) who would like to change behaviours for nominations and reviewing must get a better handle on why people choose to do either. And until we have that insight we are clutching at straws as to those drivers and it is frustrating to have to deal with it.
So to summarise
Better nominations - less appeals
Better reviewing - less appeals
Easy to say. Hard to do with an open system with few hard rules and where individual desires drive behaviours in nominations and reviewing.
😀 I love this stuff. In my job discovery is journey to better experiences.
I agree.
A good system would see few failures and few erroneous failures (upheld appeals)
Education is at the heart of it of it all, both at the start and ongoing. A you’ve been naughty email does nothing to help the reviewer improve. So it achieves nothing.
Everyone thinks their nomination was wrongly denied and yet we each will reject other people’s nominations. I think there needs to be more understanding of the fact that it’s about judgements and as such we will sometimes disagree and not take it as a personal insult. But Somehow the set up makes it feel adversarial when it should feel more reflective…..a big ask when it’s immersed in gaming which often encourages a competitive response.
Only works imo if the map is not Empty. Thus I would say your idea should be tweaked to allow for exceptions in cells with less than 2 waypoints. For instance.
What you are describing is not the problem, they are mere symptoms of the actual problem. The real problem is that the understanding of the criteria among people is just poor. Poor understanding leads to bad rejections (problem 2), and appeals of rejected bad nominations (problem 3). Problems 2 and 3 then lead to Niantic being overwhelmed (problem 1).
What you are proposing is just trying to fight the symptoms, causing even more problems in the process. Not everyone likes reviewing. I think your assumption that people who submit a lot also review a lot is wrong. I think submitting and reviewing attract different kinds of people. Rewarding getting agreements (this includes the already existing upgrades) leads to people trying to get as many agreements as possible in the shortest amount of time. The fastest way to get through a review is finding a reason to reject it and moving on to the next one. This leads to more bad rejections and more need for appeals.
Here are some things I propose:
- Everyone who interacts with Wayfarer, both submitters and reviewers, must retake the test every so often (somewhere between 30 days and 6 months). If they fail, they cannot retry the test for some time, so it can not be bruteforced. They can also no longer submit or review as long as they haven't succeeded.
- The Wayfarer test should have a much larger pool of questions, so that you can't look up the answers easily online. You should show that you actually understand the criteria. There should be emphasis on topics that are often misunderstood (these topics should reveal themselves by which questions people regularly fail, and what kinds of nominations are often appealed). After completing the test, it should actually tell you which questions you got right, and which you got wrong (and what the correct answer was), so you can actually learn something from it.
- The review page should be revamped to actually reflect the criteria. Is this a great place to explore? Is this a great place to exercise? Is this a great place to be social with others? Is it a physical, tangible object or object that placemarks an area? ... So that people are actually reminded of what the criteria are when they are actually reviewing. I am looking forward to the September update to the reviewing page. I hope it doesn't get postponed again (they have been promising this for at least 3 years), and it is actually intuitive.
- When you nominate something, it should first go to your personal review queue. You have to review it yourself first (with the revamped review page), and in the end, if you feel like it doesn't meet criteria, you can withdraw it without it costing you a nomination. If you feel that it does meet criteria, you can submit it, and that is when your "nomination" is consumed.
- Whenever a "honeypot" reaches it conclusion, send an email to everyone who reviewed it that the nomination was used as a honeypot, what they voted on it, and what Niantic thinks you should have voted. Again with the idea to actually learn something from it.
- Whenever an appeal gets reviewed, send an email to everyone who reviewed it that the nomination was appealed, what they voted on it, and what Niantic thinks you should have voted. Again with the idea to actually learn something from it.
- Get rid of any incentive to farm agreements. We only want people to review who are motivated to do it right. Reward people when they correctly vote on a honeypot with something like an upgrade. Incentivize to review according to the criteria instead of like your local community.
Bonus to speed up review times:
- Put less emphasis on city vs rural areas. Rural areas should be faster, but the differences appear to be to big now.
- Put a limit on how many undecided nominations people can have, not on how many they can submit. If you have a 2 year wait time in a city, why keep adding to the problem by keeping submitting things? Let what is submitted resolve before you can add more. Chances are that 3 other people have also submitted that same mural, and now it has to be reviewed 4 times.
@SethMizudori-PGO Can you elaborate on why my suggestion would only work in areas where the map isn't empty, please? I'm trying to see the cause-effect relationship there and it's not obvious to me.
The whole nomination process needs revamping.
When someone goes to nominate, they're excited about the thing! Their mind is on what they're going to name it. So, ask the title first.
Second: ask what eligibility category/s it meets: explore, socialize, exercise - and you can tap or drop down to get an explanation of each.
Then description, supporting picture and text, and LAST location. So they've gotten everything out of their head and can concentrate on putting the pin in the correct place, instead of rushing thru it to get to the exciting part.
The "What is it" options could be green, yellow, or red, for "usually good wayspot", "risky wayspot", or "usually bad wayspot". Or, if this can't be done, at least put a sentence saying these are NOT ALL VALID.
The review process should show what the nominator submitted, then ask what eligibility category it meets: explore, socialize, exercise - and you can tap or drop down to get an explanation of each - with a 4th option for "this meets no eligibility criteria".
Appeals are mostly only needed as a by-product of bad voting to begin with. It would be better to try and improve the bad initial rejections than have to rely on appeals as a sticky plaster fix after.
I'm all for more appeals but against having to work for them. Why should bad, abusive, pedantic voters force people to do even more work to get their submissions through? People are already having to use up their valuable submissions re-submitting wayspots.
Maybe the more you submit, the more appeals you should get. With Niantic looking at submissions marked as fake/abusive more quickly, the bots and bad submitters can be stopped too.
Having only 1 appeal fail (wrongly of course) out of 21 appeals, I am at least more confident in Niantic reviewing appeals than actual wayfarer voters.
People are giving advice in the forum to everyone telling them their submission has faults when they don't, even to seasoned submitters who have created hundreds of wayspots, when it's the reviewer's at fault, not the submitter or submission. It's senseless and the voting problems and abuse need serious looking at.
We can't rely on appeals when appeals are so slow. Speed up appeals and give me more in the mean time but still fix the root of the problem, bad, abusive and pedantic voters.
I fuller map should be penalised in the que or in your case access.
For me I have like 26 nominations available. Cut that down to like 5, however when you nominate in a cell with less than 2 waypoints your available nominations remains at 5.
Abuse button on a rejected nomination would be a good start. But how would Niantic deal with it? Send the nomination internal to great reviewers, is one way. Random rejection can the be flagged if not chosen, rejection could be upheld or overturned. A pre-Appeal check like this could cut the need for Appeals.
Agree with Appeals like to an Upgrade system. But the average reviews per month should be a gage, on how many reviews are required for a new Appeal slot. However all Wayfarers should start with like 2 Appeals, that is send back after Niantic successfully agreed with the Wayfarer in the Appeal of the pre-checked and twice rejected nomination.
@SethMizudori-PGO Ah, I see where you're coming from although I don't completely agree. Also, it would depend heavily on what size cell you're talking about. I don't think anyone should be penalized for submitting things in an area that is already populated because there are still interesting things in those areas. My neighborhood is pretty dense but it's also a hub of really fabulous street art and new murals show up constantly.
Requiring everyone to earn their submissions would allow Niantic to balance the queues, because they could tie the number of agreements required to earn a submission to the size of the queues. If there's a backlog then it would require more agreements to get one submission, but if the queues were flowing well then it would take fewer agreements.
At the end of the day generally I agree less nomination capacity. The frills of trying to give areas with less density advantage is already in the system, a small nudge to a bit more advantage will have no effect if no one nominates in those cells. Size of the cell and the number of waypoints could vary, since I have no stats to support this no good waypoint number. But generally on cell size nothing larger than the Pokémon Go ex-raid cell and nothing smaller than the s2 cells, imo.
...back to Appeals, and the discussion on limiting nominations per wayfarer. A way too encourage active nominators is to increase the nomination capacity weighted down by the number of Appeals the particular wayfarer has gotten Approved.
@SethMizudori-PGO I don't think it makes sense to tie capacity to anything about appeals. That would penalize people who create high-quality candidates that go through without needing to be appealed. Using myself as an example: Nearly everything I've submitted in the last three years has been approved on the first try. I've only appealed three things in the history of appeals. Tying capacity to appeals feels like it would penalize me for doing a good job with my initial submissions.
FYI, "Smaller than the s2 cells" doesn't work as a sentence. L0 S2 cells cover around 1/6 of the earth's surface and L22-L30 cells are measured in centimeters or millimeters.
@Hosette-ING you are basing your views on how things work in your area. Other areas don't have the luxury of submissions going through first time. And before you say it, it's nothing to do with quality of the submissions.
As someone with all but one of their appeals approved, showing bad initial voting in my area by wayfarers, I would welcome a system based on how many appeals get overturned.
It also stops or slow downs people who just keep re-appealing bad stuff that Niantic spend time rejecting again and again
If you nominate a lot and Abusive voting declines comes your way, and you are vindicated in my suggestion by International Wayfarers or Niantic. Then why should you not get more nominations? The carrot and stick approach has been considered or eluded to in various Niantic Posts: 1) reward wayfarers for voting (in game item rewards) 2) abusive voting should be handled like any other abuse... warning email followed by escalation. Now if Niantic should accept your idea to limit nominations per Wayfarer why not get a reward for nominating good stuff while pointing out abuse? Since this surely could help mold a system that does not need Appeals, Upgrades and Warning E-mails ect.
On the cells I am sure you know how (sorry for the Pokémon reference) how gyms and ex-raid gyms are formed? That is the size of the cells that I refer too. But you surely should agree if no one nominates in a particular empty cell then there is no advantage to distribute?
@RandomExploit-ING For purposes of the comment above, "my area" includes several US states in the east, west, and central parts of the country plus the UK, France, and I think Switzerland but I'd have to double-check my records on that one. For 2023 alone my submissions are split about evenly between my home area (the San Francisco bay area) and other parts of the US.
@SethMizudori-PGO I'm aware of the way that PoGo uses cells but I don't pay attention because I submit things that I find interesting and don't pay attention to how different games will or won't use the wayspot. I definitely don't think that incentives should be tied to the inclusion rules of any specific Niantic game.
I was just trying to be sensitive for using the cell concept that is not used in Ingress. Obviously with 3k posts you know what is to know.
@SethMizudori-PGO Ingress uses S2 cells visibly for region scores, and behind the scenes for MU calculations and other stuff.
Back to the pure discussion. So imo you get X nominations (like 5). If you get a Rejection you have (X - 1), if you get an Approval you gave (X). If your rejection and approval nomination screen you have a Report Abuse button. The nomination is send outside internationally to (Great or Great +1). The minority votes gets Flagged for possible Voting Abuse (in comparison with international reviews), normal Abuse guidelines are followed (statistical determinations of these multiple flags = warning email). Followed by other escalation. Niantic Appeal becomes available after International pre-Appeal review.
This structure could remove the need for Upgrades. In game rewards are linked to a system similar to Upgrades. A combination of reviews and Approvals can unlock another Nomination slot. Niantic Appeal works similar to nomination. Thus you start with Y, Accepted your (Y) are reinstated, if rejected you have (Y -1). A combination of Reviews and Niantic Approvals can unlock another Niantic Appeal, due to the hopeful success of pre-Appeal. This is more like typically achieved in 2-3 months, (on average).
Just to not miss the cell discussion. When a nomination is logged in a empty cell the nominations should remain at (X), however if rejected it should follow (X - 1).
@SethMizudori-PGO If it was me it would be much simpler.
You start with X Base Submissions per month, like low single digits but let's arbitrarily set X=4. For every N agreements you rack up you get one Bonus Submission slot, where N would be tied to the size of the queue backlog but my guess is that it would vary between something like 20 and 50. The Base Submission slots would be like they are now, they refresh a fixed period of time after they are used and they are not cumulative. The Bonus Submission slots could accumulate up to a total of Z, where Z might be something like 100. That would allow someone to accumulate a lot of extra submissions before going on a trip, or something like that. Base Submissions would always be used first. The bottom line is that everyone would be allowed to submit roughly one thing per week, and they could earn and bank additional submissions by reviewing well.
I suppose it would be reasonable to cash in your agreement rewards either for additional submission slots or for appeals, though perhaps not at the same exchange rate.
I'm struggling to find a reason to like your Report Abuse methodology. One problem is that many things are judgement calls, and there are tons of cases where something is rejected the first time and accepted the second and neither decision is objectively right or wrong. Members of voting cabals probably all have great ratings because they conspire to vote in unison and thus should have very high agreement rates... so relying on people in great status might mean that a lot of the second-tier reviewers are systemic abusers. There is also the perpetual problem that the first group of reviewers might have different reference information than the second, which happens a lot with new construction. This is one of the reasons I don't think reviewers should be penalized when their rejection is overturned on appeal... it's entirely possible for both the original rejection and the acceptance on appeal to be correct because the map data have changed between the original review and the appeal review.
I grok the desire to penalize reviewers who reject things "incorrectly", but I really think that warnings and other penalties should be driven solely by Niantic's internal data. To some extent the rating system takes care of this already because people who consistently vote differently than the majority will see their ratings drop and their votes will have less weight. Niantic has the big data available to them that will permit them to identify voting cabals and other irregularities systemically, and that will have a much more reliable signal than player reports. (I have some professional expertise in using big data to identify fraud/abuse, although it was a decade ago and the tools and methodologies have progressed far beyond the things I was able to do then.)
I agree with Z (bonus nomination max), not N (for me the que should not be a factor, bonus and wayfarer events and Niantic can deal with que issues as before). I do think like every month or 3 one nomination should reset. Also in game rewards should help to encourage more wayfarers, tied up with agreements similar too Upgrades.
Abusive voting is a thing. Then maybe the international pre-Appeal voting should see all the listed rejections, when a nomination is rejected. And have the ability to flag rejection reasons used as bogus. Group think is what is desirable for reject or approve, new anti-abuse voter ring systems/measures can weed out voter-ring bias.
How can you and others not see that nominating and reviewing are 2 different things that should not be linked together? If I'm not mistaking, you have said somewhere that you have dozens of upgrades stacked because you review a lot but don't submit much, so you must be aware that submitting and reviewing are different. We should not force one group to do what the other group does. Forcing one group to do what the other group does against their will, will cause them to do it with low motivation to do it right, leading to lower quality.
Also, the concept of agreements with the community is bad and needs to go. It does not encourage you to vote according to the criteria, it encourages you to vote like others in your local area. My local area is heavily under the influence of the Dutch bots. If I want agreements, I have to accept everything, but then I risk that my Wayfarer and game accounts get banned. If I vote according to the criteria, I will get fewer agreements and my rating will drop. So for now all I can do is not vote at all. How can I earn more submissions now? How long before the abuse in my area is gone? Germany has the opposite abuse were everything is rejected by bots or cabals. That has been going on for years and as far as I know there has been no improvement made on that front. We should not get rewarded for agreeing with the community, only with Niantic. So only agreeing with honeypots should get you rewards and should affect your rating.
And it is not because your area is not affected by bots, that my problem doesn't apply to you. Every area in the world is some kind of a cabal. Every community has their own made up rules that they don't even realize are made up. In the USA for example, the community has made up the rule that private residential property extends to the street. This is totally made up by the community, yet many US residents even on this forum act like it is gospel and won't listen to any arguments that it is not. So people accepting nominations in the US that are in front of but clearly outside of PRP, will not get an agreement, despite making the correct decision. People rejecting those nominations will be rewarded, despite not following the criteria.
It is interesting to read these suggestions, but it also hurts the brain cells 🤪
If suggestions are to be made they need to be very simple, as this aids transparency.