2021 Dec. AMA - Response Discussion
NianticTintino Posts: 874Niantic › admin
edited December 2021 in December AMA - 2021
2021 December AMA responses are here!!!!!! Thank you all for participating and for asking all of your questions. We selected 15 of the top voted questions and added an additional one as a little something for the end of the year. The responses you see below have been mainly organized in most-voted order with the exception of a few. As you can see below, the bolded text includes the discussion topic title followed by a summary of the question in italicized text.
Without further ado, here are our responses:
- What does the internal review process look like? - Can you give us any insights into the internal review process?
- Our internal review staff go through an intensive training on the Wayfarer criteria and we do recalibration sessions as needed. They have a different internal tool to review nominations, edits, and abuse reports. We do realize that sometimes our team gets it wrong and so we are ramping up the recalibration sessions with the team.
- Centralized Criteria Please - Can we please get a centralized repository of feedback, clarifications, and overall criteria judgements, so that researching eligibility is easier for everyone?
- Thanks for asking this. This topic is one that I hope to tackle in the new year. I myself have been looking and thinking about better ways of communicating this on the Wayfarer website but also in the Wayfarer Community. For the main central repository, that will be in the Wayfarer review portal, under the tab called Criteria. As we add, change, remove, etc it’ll be updated and also communicated in the community. There’s definitely some work to be done to make it all easier for everyone. When it comes to the Wayfarer Community, I’ve thought of ways of reconfiguring the Community Forum so that feedback, clarifications, etc. have a respective place to live but also be visible and easy to find. We will continue to flesh this out and this will be a part of our next year's work.
- Why do we need AMAs to answer these burning questions? - How do you see these AMAs evolving over time and will they be our only source of clarifications and answers to issues?
- As mentioned in the 2021 Sept AMA, I hope to make AMA’s more fun and cover different types of questions but also still leave space for deep questions about features, processes, etc. that may not always get answered or questions that develop through time. I think we’re still a ways away from an ideal but also something we’ll continue to work on for next year. While this AMA had some minor changes to it, I hope that they continue to change in ways that help all Explorers. What do you think AMA’s should be used for?
- What is being done about existing nominators/reviewers not keeping up with criteria? - How is Niantic trying to find a way for existing nominators and reviewers to actually look at and follow criteria instead of these popular but now ineligible wayspots?
- This is something that we’ve also been thinking about and will continue to work on. I’m excited to put my educator hat on and think critically about better ways of creating opportunities for Explorers (new and experienced) to touch up on their knowledge of the criteria. One way you’ve seen this is with the new onboarding flow, rejection reasons in the emails, etc.
- Is there any possibility of streams or videos of Niantic reviewers going through the review process? - Are videos or streams of this nature a possibility?
- This one is super exciting and I’ll ask that you all stay tuned for a little while longer (like a month or two) while we finalize a cool new… “something”. Similar to the previous question, we’re taking a good look at how we can help Explorers touch up on their criteria knowledge and also be a type of guide for new Explorers. All to say, we’re excited about what’s coming!
- What is the purpose of 2 and 4 star ratings? - What should the 2 star and 4 star ratings be used for?
- That is a great question - Initially the thought was to give you a range of response options to questions that are frankly pretty subjective in nature. But we are thinking about changing the question structure next year. A response scale doesn’t always fit the questions we’re asking. Some should simply be a yes/no. And star ratings shouldn’t be used as a proxy for rejections or ‘I don’t know’. So in short, they haven’t really been working the way we intended them to and they’ll be changing next year.
- 30-50 to approve vs 5 to reject rumor - Can you confirm/deny these numbers, or comment on any plans on how you combat rejection-agreement abuse?
- It takes a variable number of reviews on a contribution before we decide to take the majority-wins decision, a number of which is dependent on your Wayfinder rating among other things. It seems the real problem here is spamming of responses in order to earn Upgrades. For any cheating behavior, it always takes a multi-pronged approach. Besides punishments, we want to create new fun ways to earn Upgrades that don’t push you to spam your way to getting that Upgrade.
- A way to edit Lightship POI’s - What can be done to see/edit Wayspots that don’t happen to show up in Ingress or Pokémon Go?
- Hey, glad you asked that question. As @NianticDanbocat posted this summer, we are thinking about what a Wayfarer app might look like but we’re still very early in the discussions. We know that having the Wayfarer contribution features in games does limit us in terms of what features we can add or information we can show you as a contributor, so we’re really interested in the concept of a Wayfarer app to solve this problem for you.
- Appeal System - Can the current appeals process be changed?
- This one is interesting to answer. So to be honest, we’ll be keeping it as is for the next couple of months (~6 months) before we make any changes. We do this to ensure we do things the right way and allow the current process time to stabilize. We’ll be looking at our own internal processes and monitoring how the appeals system is performing before making improvements.
- New Wayfarer Challenge where the whole community can participate or other forms of competition - Will we be seeing another Wayfarer Challenge any time soon?
- Yes, definitely. We have been taking some time to rethink how we do Wayfarer Challenges. We are playing around with some ideas that would center our mission of exploration, allow folks to be social, and to get out there and explore! Let’s make sure we step away from the computer and have fun during these challenges! In addition, we’re also going to be thinking about new ways to reward. It’s likely that new event formats will focus on specific cities aligned with our 2022 Niantic company goals.
- Eligibility of parks with no traditional physical markers - If there is a documented park but the park had none of the normal Wayspot objects would this park still be eligible?
- Alright, so yes, they’re still eligible but just make sure you take a photo of the park where: the photo makes the park identifiable to someone who has not been there before. Try capturing enough of the scene so that as someone is walking towards the destination, it’s easy to spot. Note that we plan to restructure the rejection reasons next year, as some no longer apply with the criteria refresh we did a year ago. For example, a rejection simply for being generic isn’t sufficient anymore, what is important is how interesting and relevant it is as a place worthy of adventuring out to.
- Allow changing Bonus Location more often - Are there any plans to change the current policy on Bonus Location currently only being changeable annually?
- Good question. We know that there are still ‘dry voting areas’, and that dedicated contributors such as yourself would be willing to help out other communities in mapping out their locations. We certainly think there are better ways to enable good honest reviewers to do this and are thinking of a new Explorer ranking system and different rewarding mechanism that helps here. On the question about upgrades - no upgrades are not viewed as a tradeoff between accuracy and speed of resolution. Upgrades reflect how aligned your decisions are with the community only.
- Why do Good/Great reviewers get less frequent local Reviews than Poor/Fair? - Why is it set up that Good/Great reviewers receive the majority of their reviews from outside their reviewing range, while Poor/Fair get mostly from their own area?
- That’s an interesting idea. There are certainly improvements we can make on how the contributions are selected for you to review, one of which was mentioned above about enabling good honest reviewers to help out in dry voting areas outside their local community. There’s ideas brewing around building a ranking system to allow certain Explorers access to more custom ways of selecting where you vote, but adding a voting weight to it is interesting too. We’ll think about it!
- Further clarification on rejections for "not visually unique" or "not historic/cultural” - Why is this listed as a rejection reason if the rating for this category does not affect the final decision of a wayspot?
- Yes, we will investigate and look into this, but know that these have never been valid rejection reasons in the past. The scores were to help us differentiate between a good/ok candidate (exercise machine) vs a great candidate (Art/Historical locations). As mentioned above, we’re working on reconciliation across the Wayfarer review portal, the in-game report rejection reasons.
- How can we actually get Niantic’s attention? - What's the best way to get attention to issues and actually have a conversation about what's going on?
- Thank you for sharing your frustration around connecting with us. For now, please use our existing in-game support channels and official communities where available. Even when you don't receive a response, know that we are monitoring the feedback and will continue to improve our support. The support teams are mobilizing internally to improve overall support across Niantic products in 2022. This is a project well beyond just the Wayfarer community, I’ll be doing my best to ensure your support needs are brought to the table.
End of Year Bonus Question!
- Disable all contribution features before completing the Wayfarer test - Are you still planning to change how new users get access to the in-game contribution features?
- Yes, we are planning to ship this in Pokémon GO and Ingress in January. Yes, once shipped, all contribution features will be blocked in-game until you go through the onboarding flow and ultimately pass the quiz in Wayfarer. We’re also reconsidering how players are gated from accessing Wayfarer that is perhaps not level gated, but gated by a rank or “trustworthiness” score that you build up as you learn different parts of the system for community mapping.
Happy Holidays and New Year!
Awesome stuff! About the bonus question, if you know and can tell us the answer: I believe Scans in Ingress and PokemonGO are also classified as "Contributions". Will those also be locked behind the Wayfarer onboarding, or will scans remain behind the normal (in game) level gate they have now, as it doesn't really change the state of the Wayspot itself?
Also for the bonus question - I have a friend who currently logs into Pokemon Go and Ingress through his Apple ID, which is allowed but isn't currently a Wayfarer login method. Will he (and others like him) be able to submit waypoints at some point in the future after the new contribution features have shipped, or will this eliminate his ability to do so?
Mostly good answers, was kind of hoping we would get q rough idea on whe appeals qere expected to start coming g through, but i suppose, they must expect them to start going through in less than 6 months 😅
Hopefully with them saying they will look at historical and visual, those will no longer cause a waypoint to unfairly fail
Good to see the 5 star ratings system will most likely be gone soon, will remove the confusion over whether 3 star is an accept 2 star a rejection or an accept etc.
Yeah I've been seeing historical/visual rejections since the launch of Wayfarer, so if that wasn't intended as a rejection reason that was a pretty bad design on the part of whoever didn't catch that it was a reason.
Wait, where is my question about cemeteries? Its #13 but not in here :(
Uhm... Didn't the answer to question #13 completely miss the actual question?
Could we get an answer to the question #13? It seems it was misunderstood.
Edit: sorry, didn't update the page and missed the post above mine.
Disable all contribution features before completing the Wayfarer test - Are you still planning to change how new users get access to the in-game contribution features?
Yes, we are planning to ship this in Pokémon GO and Ingress in January. Yes, once shipped, all contribution features will be blocked in-game until you go through the onboarding flow and ultimately pass the quiz in Wayfarer. We’re also reconsidering how players are gated from accessing Wayfarer that is perhaps not level gated, but gated by a rank or “trustworthiness” score that you build up as you learn different parts of the system for community mapping.
My love, my love, all my love for including this as a bonus question even if it didn't get the allotted votes, my heart thanks you for the confirmation. It is also true, that accessing Wayfarer should not be level-gated either as XP/AP is not an indicator for whether someone will make good edits or not (judging from the coal that comes around and the bad edits).
This means that:
It's not yet January, but I pray this will mean the end of the ugly photo abuse and all that ugliness that happens around Circular Quay in Sydney. I had to review a photo of someone's thigh, a worm, and trash on a street very recently!
I am grateful of this upcoming feature, but I must eagerly await the day you ship this. Again, highly grateful that this was a bonus question. Merry Christmas Niantic and the Wayfarer team! In Australia, coal is very much the golden mineral desired in everyone's Christmas stocking!
Regarding the following part of the answer of #11; "For example, a rejection simply for being generic isn’t sufficient anymore, what is important is how interesting and relevant it is as a place worthy of adventuring out to. "
This comment was written in a way that lacked the perspective of "A great place to be social with others" and should not have been included, at least not in the response to this item.
Is a place with just a lawn, for small children to play, worthy of exploration by outsiders?
However, such places are also valuable social gatherings for the community.
It's a shame that such an answer was presented, even though a very good example was presented in the original thread.
Please report back in six months how many appeals you have reviewed, along with the number of backlogs. I'm very interested.
Some thoughts on this last one;
Yes, once shipped, all contribution features will be blocked in-game until you go through the onboarding flow and ultimately pass the quiz in Wayfarer.
Not saying we should go back to two attempts and then failed forever but I don’t think the 15 minute thing will work. People will just brute force it.
Surprised there's so little engagement on this, maybe need to move it to general chat?
The quiz simply makes sure the potential reviewer/nominator understands basic concepts. It could perhaps stand to be more elaborate and comprehensive, but for an ever evolving platform and community, an overwhelming endeavor to work out all kinks in the initial quiz alone would, honestly, probably prove futile over time. This is where forums come in. :) Do you have a suggestion to accompany your critique, or an explanation of why asking those unacquainted with Wayfarer to take a quiz prior to submitting wouldn't be beneficial? "Brute forcing" would be an outlier concern, I think; ultimately, this would cut down on "coal" submissions that are often the focus of exhausting complaints.
There are two main objectives to this:
I am going to take a wild stab and say most abusers are illiterate. Well, the Sydney ones that is.
To be fair, it is the festive period, people are probably enjoying time away from the forum haha
To be fair, there wasnt much in it.
I didn't wanna be the one to say it but... yeah. Nearly the entire assortment of questions and answers treads well worn ground, with very little in the way of compelling new information.
It wouldnt be fair to the team that does good work compared to what happened her before. We dont really need whitewashed AMAs that circle around the answers we are looking for. Ask me anything? Yes, but we will answer the questions we like in the way we are told.
Love the bonus answer, there should also be quarterly re-calibrations sessions. I also agree that the level gate should be removed, great idea !
I just want to ping about one particular answer that may need more attention, specifically #11.
"11. Eligibility of parks with no traditional physical markers - If there is a documented park but the park had none of the normal Wayspot objects would this park still be eligible?
a. Alright, so yes, they’re still eligible but just make sure you take a photo of the park where: the photo makes the park identifiable to someone who has not been there before. Try capturing enough of the scene so that as someone is walking towards the destination, it’s easy to spot. Note that we plan to restructure the rejection reasons next year, as some no longer apply with the criteria refresh we did a year ago. For example, a rejection simply for being generic isn’t sufficient anymore, what is important is how interesting and relevant it is as a place worthy of adventuring out to."
In my city, there aren't a LOT of such (as I call them) "parkettes" -- generally just a bit of green space (whether flowerbeds or maintained grass or both), almost always with some seating. Most don't have "name of park" signs, the ones that do were all already wayspots. A few have other plaques or markers, and those objects were usually already wayspots (often in lieu of the park/parkette itself).
But there's still a few spots that are nothing but the seating and the greenspace. And to date, the only one of those that have successfully passed Wayfarer and are now wayspots was the one manually reviewed by Niantic. There are still a few others around (I know of at least three), but none of them are photogenic enough to bother with until spring comes around again. Hopefully by then the reviewers will have finally gotten the memo.
Appeals - all that's been said about them is that you're going to keep doing what you're doing for six months. My question is - what are you doing? Have you processed any appeals at all? Have you added additional staff to handle the influx of appeals? I don't have much to appeal so I have no idea - has the appeal limit been rolled back to the stated One Every 30 Days level or are you still allowing one per day, and if so why? If you are indeed processing appeals, what is the current rate per day or per week? I know you can't comment on exactly how long it will take to get a decision on a particular appeal, since that's based on several dynamic factors. From the reports on this forum, though, I'm not certain you've gotten to anyone's appeal at all. Should we just expect six months more of inaction?
YES! It would be very helpful to have more transparency on this. If it is going to take a while there also needs to be an option to rescind, otherwise there could be duplicates and we are just clogging the queue. If these are not timely, it's not super helpful.
In reference to Question 14.
I've had several small play areas / single piece of play equipment (ie balancing beams) which btw have a small official sign and classed on the resident estate as technically play areas, REJECTED due to the 'Not Historically or Culturally Relevant' Rejected Reason...
Well of course it's not Historicial or cultural, it's a play area/exercise equipment. >< One of the three core criteria of eligibility that are sort after.
So it kind of has a disadvantage from that part of the review from the other two main eligibility criteria.
I feel that very often judgments are made by many judges based on "old standards" or "personal values".
When a new AMA is issued, the existing judges should be tested according to the new AMA, even if they have to stop judging temporarily.
@NianticTintino In reference to your answer to Q14 (rejections for "not visually unique" and "not historically/culturally significant", has there been any progress with this investigation?
Aa member in my community today had a basketball hoop rejected due to not being historically or culturally significant, so there is still a discrepancy between has what been said publicly and what appears to be happening in terms of Wayspot rejections.
I can also confirm that this has been an ongoing issue over a number of years. The following Wayspot nominations of mine have been rejected over the last few years with only historical/cultural significant or visual uniqueness listed as the rejection reason.
"Not historically or culturally significant"
"Not visually unique"
Adding another one to the above list: Had a park playground today rejected for historical/cultural significance. 😕
Are you still planning to change how new users get access to the in-game contribution features?
Yes, we are planning to ship this in Pokémon GO and Ingress in January. Yes, once shipped, all contribution features will be blocked in-game until you go through the onboarding flow and ultimately pass the quiz in Wayfarer.
This seriously cannot come quicker enough. I am tormented by all the fake things that I am reviewing everyday, from ugly one-eyed babies (literally) to severe fakes and third-party images.
While we won't see immediate changes because such photos are still being filtered out of the queues... it's best that we start as soon as possible.
So glad to see a response to #4 about submitters and reviewers not keeping up with criteria. I’ve submitted several new stops because of the new criteria but they get rejected because reviewers are still thinking about the old criteria. Make us all retest so we can get through this.