What is being done to increase the speed of processing Appeals? Six months is way too long.
Griffinfinity-ING
Posts: 48 ✭✭✭✭
Currently, Appeals from late April/early May are being processed. What is being done to increase the process speed and not having submitters wait more than six months to receive a resolution from their Appeals? Or do you think waiting six months is an appropriate wait time?
Comments
As long as the Wayfarer team is manually reviewing them in their own spare time, it is unlikely to change.
The same thing is currently happening as Niantic stopped accepting nominations once in 2015.
At that time, it was said that it would be six months or a year before we would see the results of the nominations.
And they are implementing OPR to resolve this situation.
So when the Wayfarer team implemented the appeal system, I assumed that they would be using artificial intelligence to make automated decisions or outsourcing the review process.
However, the reality was different.
I still have an appeal waiting from January and another from February. Far more than "six months".
Niantic may not want to tell us WHAT is being done - every company has its "secret sauce" - they don't give out all their technical information to the public.
I think the questions are:
Is anything being done to increase appeal process speed? How much elapsed time does Niantic see as acceptable?
It's infuriating that our only reward for reviewing is upgrades which are supposed to give us a quick resolution, only for it to be rejected for nonsense and then wait months for Niantic to correct it. I think that appeals for upgraded submissions should be a priority at least, or there is little incentive to review for upgrades.
A side effect is that I now sometimes use my upgrades on submissions that I know are eligible but think the community is going to reject, just so I can submit the appeal quicker, when I should be using them to prioritise the best wayspots.
There was no progress on appeals until June or July. Something went wrong (code didn't work, or something) setting up the initial system. That happens.
However, if we ignore that initial startup glitch and start counting with June - it's still taken really long. Seems like we could be more caught up with the backlog by now.
Also, the appeal-received email says they will be processed in the order received, but they are not.
How about a Niantic goal to halve the number of outstanding appeals by New Year's? Oldest first.
I am sorry, but some of your information is not correct.
That is not true that there was no progress on appeals until June or July. I have appeal decision emails from back in March.
Appeals are mainly processed as first in first out. Towards, the beginning, it was pretty random, but I think the FIFO started in April. There are times where the FIFO break & randomly ones get pulled in. (I am in a Wayfarer discord, and it definitely seems to be FIFO from what I noticed and discussing with everyone else there.)
OK it just seeeemed like six months before any voting, but it was "only" three.
I still have an appeal waiting that I made in January, and one from February. Yet other people say they've processed thru May.
My old appeals have been "In Voting" for a month or two. Maybe everything thru May has moved to Niantic "In Voting" - but there's no sequencing after that, and some of us have bad luck. Both "In Queue" and "In Voting" are not resolved.
It's frustrating, however you dissect it.
May I suggest the following?:
If the Niantic staff is overwhelmed by the amount of appeals, perhaps the team could consider:
This could consolidate and categorized the Appeals, and made the Niantic's staff's review for the Appeals easier.
Of course, Niantic still hold the final decision in determining the fate of an Appeal.
Great suggestion!
Problem with this system is that every suggestion has a downside. My area has a dozen or so great reviewers. But our (un upgraded) nominations take months to over a year to process. I wouldn't want them to review more somewhere else, so less locally, so it takes even longer to get to my noms thru.
Just going to leave this one here.
They did answer this to some extent in the April 2022 AMAs; they had processed 20k appeals within the four months of it being active, and said they needed 150k processed appeals to begin their "analysis". I don't believe we're anywhere remotely close since appeals around May are only being pushed into voting now.
Their "analysis" means they're training AI (Artificial in Intelligence). The 150,000 appeal goal is LOW for AI input, but they'll probably also feed it results from 2021's regular Wayfaring (now that it's had a year to be ironed out by appeals and reports). As I recall 2021 had around 1.5 million accepted, plus, guessing another million or so rejected.
AI requires millions of binary examples (in this case, nomination/result). You always use 80% of your collected data to train the AI, and save 20% for testing (10%, then tweak the code, then 10% to finish). (If it needs more tweaking, you need more data.)
AI results always come with a confidence level. I'm guessing Niantic will use AI to get the easy ones. Like, if it's 90% confident it's an accept or reject - let it on thru. Anything it's less confident about - send to humans. If you've ever studied traffic (computer or automobiles) you know that slicing 10% off a clogged area can reduce its wait time by like 50%.
You want "fresh" data to train AI, so they wouldn't outright feed it nominatons/results from, say, 2017 - but an appeal from 2017 that was resolved in 2022 would probably count as "fresh" because it was recently re-reviewed.
👀
@PkmnTrainerJ-ING Ok, that's why they slowed down on appeals. The AI/ML team isn't even built yet, to use the results.
Great find!
Know what - that also explains why so many things suddenly started going straight to Niantic Voting. It was things the AI was getting wrong, so they needed more test data with the words that confused the AI.
I think the best thing we as a community could do is identify the Duplicates to get them out of the system. Heck, we could even ask people on this forum to submit all their duplicates to a thread so they can be pulled out or just processed immediately. For clarity, a duplicate happens when a legit submission is rejected by the community so the person appeals it. But then appeals are so bogged down and slow that the person just resubmits it and the local community approves it before Niantic gets to the appeal. I've had that happen a LOT.
Even better would be if they would let us withdraw appeals. Then we could remove our own ones that have been approved in the interim and not waste everyone's time.
I would welcome being able to withdraw an appeal.
i have 2 in appeal that have now been approved via the actions to clear the backlog. That is 2 out of my current 7 appeals so a significant proportion.
1 appeal has already been resolved as a duplicate because of this scenario.
anyone due to be able to submit a new appeal but the button isn't showing up to be used?
It took me entirely too long to locate this and as noted before, it may not be relevant to Wayfarer even, but to follow up on the job posting;
It seems that the team is still growing at least. This was taken from GlassDoor
Six months would be amazing! I still have some from the beginning of last year!