Upgrade Issues
NianticTintino-ING
Posts: 1,068Niantic › admin
Hello Explorers,
We received reports of explorers whose upgraded submissions took longer than expected to be processed. Our ambassadors provided us with examples and evidence of this behavior and escalated the incident to our team.
Following an investigation we have confirmed the issue and identified the root cause of the problem.
Certain upgraded submissions were preventing others from being reviewed, delaying the upgrade review process.
Our engineering team is working on a fix to remedy the situation, preventing other records from entering that state. But the amount of upgraded submissions had grown from this incident and still requires some time to go back to a nominal processing rate. The team has identified means to improve the speed at which those are processed.
We apologize for the inconvenience and want to thank you for your patience. We know that Upgrades are important to you and take a lot of work to earn.
We will continue to keep you posted here once the fix has been fully applied and you can expect Upgrades to behave as intended.
Update - [06/15/2023 4:50 PM PDT]
Thank you for your patience while we worked on this new fix. The Wayfarer team has applied the fix and we can see the fix having a positive impact on Upgraded resolutions. Thank you all who have already reported seeing momentum in your Upgrades as well. You can expect your upgrades to return to normal speeds in a few hours. We are mindful that this incident has impacted the usefulness of your upgrades and we are looking at a way to identify the impacted upgrades to reimburse. We will provide you with an update on that shortly once we are sure the situation is back to nominal state.
Like always, we will continue to monitor the progress and keep you updated if we see any issues with Upgrades.
Update - [07/17/2023 2:25pm PDT]
On May 23, 2023 we started experiencing issues with our Upgrade system that impacted your applied Upgrades during that time. We have been able to identify those impacted by the issue and will be reimbursing an Upgrade per upgraded nomination impacted per the parameters below:
- Your Upgraded nomination was not from one of the countries of the challenge
- which was upgraded between May 23, 2023 and June 10, 2023, OR
- which was an upgraded nomination prior to the period but resolved during the challenge or May 23, 2023 to June 10, 2023
- The resolution time of the upgrade was >2 days (the time between when it was upgraded to resolve)
These Upgrades will begin being reimbursed as soon as today and will continue being reimbursed to your Wayfarer profiles in batches on a weekly basis until they’re all delivered.
Safe Exploring!
Comments
Thank you for acknowledging this, @NianticTintino-ING. Are you able to clarify that issue is also preventing Upgraded Nominations from entering Voting status?
I currently have two nominations which I upgraded 7 and 10 days ago while they were in queue, and they went into voting normally. Of course because of this issue, they have been stuck with ”in voting + upgraded” ever since.
Thanks for keeping us update. What I understand is that you’re working on a fix for next upgrades but what about the ones that has been applied to nominations and are stuck for now ?
Will we get our upgrades back?
As the person above already mentioned, will this get us back our upgrades?
Upgrades that don't properly speed up the process and that have been wasted on submissions that have already languished for over a week should be returned, or bonus upgrades should be provided to those affected. The community works hard to build your database of locations worldwide, Niantic. We would appreciate the acknowledgement of our hard work.
Right! The steps should be
1) Fix the bug
2) Everyone gets +1 upgrade for every upgrade delayed by this bug
3) Process everything upgraded before the fix
4) Resume with processing Wayspots upgraded after the fix (not until after all the delayed ones are processed)
Thanks for the update. Feels good that you keep us informed with this issue
I feel like not using my upgrades for now. I already figured out no upgrade was faster.
Thanks for the heads up for this issue. I'll notify my local community to hold on to their upgrades until you were able to get the issue resolved.
I think I don't ask for the impossible to put a warning on the showcase page to hold on to your upgrades. Because if it will not be possible to get the stuck upgraded free automatically or give out automatic upgrade refund, it might be best to prevent a high workload for manual refunds or manual freeing the stuck subs. So it will be a win for both sides if you could put up a banner warning.
Also the possibility of the really high numbers because of the Spain Challenge. As we speak the banner tells me that in less then 90 minutes there were already 994 resolved records. So a lot of upgrades will be earned is my prediction.
Right. If upgrading would throw our nominations into perpetual limbo, Niantic would turn off upgrades until it was fixed... right?? Or are upgraded nominations just going to be rejected without review (like some really old nominations were) - and Niantic is just letting us keep throwing our hard work off a cliff?
Any update(s) on fixing this issue yet?
It seems like someone did something. My stuck upgraded submission is now in voting.
Consider yourself lucky. Twenty-two days without movement here.
Yeah, my last one strucked upgrade finally moved yesterday into voting, and today was resolved as... rejected by "Other criteria". Just this one. I lost both upgrades being rejected by just "Other criteria", when usually when I get a rejection I got 2-3 reasons to justify/explain the reasons behind the rejection. I have to think that they're actually automatically rejected in the process to purge the queue formed by their bug :(
It just got resolved. So it seems things are flowing again. I'll give it a try with an other to see if this will go through
I can also say the submission that i upgraded went in voting quick and is now resolved. It seems to have been fixed.
Quick question for people actually getting stuff through, how fast are non upgraded moms in your area? Could they be going through because the resolved at normal speed?
The only non-upgraded Wayspot I ever had go through was after they attempted queue cleanup. That was submitted in Nov '21 and closed in March '23. Have had other submissions in voting for months. Major American city.
It really depends, some are just slamdunks for everyone and they can go through in anything from 12 to 24 hours. But 3 to 7 days is more common.
And some of my submissions are going in voting the same day, some take days or sometimes weeks. (Hot summer weather is a killer for progress).
Currently i have 1 waiting and 2 in voting. They where from a string of 20 on one day. 17 already approved and are on the map or will be tomorrow. Just one was upgraded.
I'm still puzzled though, why do some later date submissions go in voting while others hang for days or weeks(or months with some people). But that's for another topic.
Just a caution to anyone thinking about using them. Both mine and my husband’s, upgraded May 9, are still stuck In Voting with no sign of any movement.
Mine was on the 15th, still in voting. My wife has one upgraded and in voting since the 2nd.
Just a thought, but if I'm not just lucky, the system might just be going from new to old to fix.
I do have to put the side node in. That i might have profited from the Spain Challenge. Though I live in the Netherlands, more people are voting and the time it takes to resolve a Sub has gone down the last week. So it might be that this issue seems to resolve quicker in Europe. Because they're barely any Subs left in Spain i now get a lot of French. That might result in more people are getting other countries in Europe as well.
But then again, I've might just have been lucky.
Well now something I submitted the day AFTER my upgraded submission has been decided on. So not only does the upgrade not speed up a decision, it appears to have actually slowed it down MORE than if it wasn't upgraded. So that's fun. Definitely not a waste of all that time spent reviewing or anything.
Thanks for the info on your submissions. Definitely jealous. It would be wonderful if we could have an update from Niantic instead of random speculation from fellow Wayfarers.
There’s an ancient saying in software development: ”If it ain’t broken, don’t fix it.”
In Niantic’s case it should be: ”If it ain’t broken, don’t break it.”
If you have a 65% agreement rate, and average one minute per review, it takes 2.5 hours to earn each upgrade.
But, it's free for Niantic. Also: It doesn't matter to them if your nomination is approved over some other nomination - their global database grows no matter which nominations are processed. Lastly: each of us are expendable. Many members of their volunteer workforce have quit thru the years, but others take their place, and there's no effort to coax anybody back.
Great perspective. This is a bad way of thinking on their part though, because this issue is impacting the #1 incentive for Wayfarers and disincentivizing long-term Wayfarers decreased the overall health and reliability of WaySpots.
We already have enough problems with bad nominations getting through, can we imagine if the quality were to decrease by even 5%?
I am now up to 4 stacked upgrades (plus whatever I get from this challenge and as I continue to review for the challenge). It is so demoralizing to not be able to use them. My nominations take over a year to resolve naturally. I’ve done over 45k reviews for this system, just let me have working upgrades back 😭
I have an upgrade that has been in voting since may 13th! and my wife has one in voting since May 14th! This is very frustrating. I upgraded it so that it would get decided on quickly! 😫
@NianticTintino-ING Update, please!