Three changes I would make if I was in charge of Wayfarer, and my reasoning
Hosette-ING
Posts: 3,470 ✭✭✭✭✭
I enjoy analyzing systems, looking for persistent problems, and thinking about what changes would improve the system and reduce or eliminate persistent issues... in fact, it's a nontrivial component of my career. Wayfarer is just a hobby, but I read a lot of posts on various forums and see a lot of common complaints about it, and I amuse myself by thinking about how I would improve the system if I was put in charge. With the disclaimer that these are based solely on the black box view, with no inside information, these are the top three changes I would make.
Change #1: Provide on-screen guidance for submitters and reviewers
Why? People are woefully undereducated about the current eligibility and rejection criteria. Human memory is highly fallible, especially when you're trying to add layers of changes to base knowledge. The submission queue is jam-packed with things that are egregiously ineligible... this is frustrating for submitters who thought they had a good candidate, and it wastes reviewers' time and slows the whole system down.
How? When someone submits a new wayspot candidate their first step would be to choose the "What is it?" category. Once they do that they would get on-screen guidance that was specific to that category, as well as broader general guidance. Submitters would also have to answer a few filtering questions like "Is this on the property of a K-12 school?" If they answered yes they would be given specific information about ineligibility.
When reviewers see a submission they would see similar on-screen guidance based on the category chosen by the submitter, though if they thought the category was wrong they could correct it and get guidance based on the updated category.
What would this accomplish? There would be less garbage in the queues because submitters could avoid submitting some of the stuff that's clearly 1*. Submitters would have less frustration from having their candidates rejected and reviewers would encounter far less coal. Reviewers would make fewer mistakes because they would have the information that they needed while reviewing and not have to rely on their memories.
Change #2: Automatically expand review areas for candidates in voting for more than X days
Why? Some areas don't have enough reviewers to get submissions reviewed and things get stuck in voting for weeks or months. Niantic has a cold-start problem with remote areas, and even if an area manages to get things submitted they won't ever be reviewed/approved without upgrades. Players have worked around this to some extent by organizing groups to set their bonus locations to a specific location, and Niantic has done location-specific challenges, but these require significant effort and only address the targeted areas... they don't solve the systemic problem.
How? I addressed this in detail in this comment, but the summary is that when a candidate has been in voting for X days the review area is expanded and the timer is reset. Keep expanding the review area every X days until the candidate reaches consensus.
What would this accomplish? Although some areas would be slower than others eventually every candidate would automatically get a large enough audience to get a decision. Players wouldn't have to organize bonus location swaps and no area would get left behind. Reviewers that are close to review deserts would see a few extra things while reviewing.
Change #3: Every candidate would require at least 51% of reviews to come from outside the area
Why? We've all heard plenty of complaints about areas with large numbers of fake wayspots, or areas where reviewers have banded together to reject everything or perhaps everything that was submitted by someone outside the group. St. Cloud, Germany, and Roseville pop immediately into my mind but I know there are others. Oh yes, there was also that town in Vietnam that magically had exactly one wayspot per L17 S2 cell including murals that were allegedly in the middle of farmers' fields. The stuff that's been reported here is probably only the tip of the iceberg.
How? There are multiple ways to accomplish this but the easiest one would be to require a majority of votes for each candidate to come from people whose home, bonus, and local play locations are all outside of the normal review range.
What would this accomplish? It would be far more difficult for local reviewers to band together (or one person to use many accounts) to manipulate their local area. We would have far fewer new fakes, and probable-botnets like the one in Germany would be rendered ineffective. Reviewers would see a wider variety of candidates.
[Note: I understand that this one will be less popular than the other two because some people have a strong preference for reviewing local things, but I believe it would be pretty effective at stopping bots and local cabals.]
Those are the top three things I would change if I was in charge of Wayfarer. What do you think? Would they result in a better system?
Comments
I would change the review process by breaking it down into three parts, first how well does the nomination rate against the three eligibility criteria on a scale of 1 to 5, then confirm whether it meets all acceptance criteria (including location accuracy, pedestrian access, and duplicates), and lastly, which rejection criteria-if any-that the nomination may fall under.
Breaking the review process along the Wayspot Criteria will allow reviewers to more accurately assess each nomination, provide more accurate rejection reasons for the submitter, and hopefully deter "speed reviewing". It will also give Niantic a better chance to determine if something is wanky with a reviewer, especially if they are frequently giving 5* in all three eligibility categories.
@Hosette-ING really like your ideas.
My fuzzy wayfarer overhall idea also centres around category.
For nomination: 1) Location 2) Category 3) photo 4) surrounding photo .... title etc. Just prior to submitting needs then to have a checklist specific to category.
But also for review: imagine if you could just check gazebos or things you felt you had a real grasp on. Also batching .... 10 gazebos in a row . As it is it's a load to jump between so many conventions. If I could just have 10 park signs to check before my train ... Rather than a park sign have they got the right name?, lfl - is it fake?, Is that old church now a prp or is it still a church?
Lol the game might end up with many many gazebos. Lol ... But is that a bad idea,?
With your last idea, combine that with a per-user preference to review submissions from outside their usual area. Some will be happy to review more widely, some won’t. Why not an option to choose?
If there are a large amount of people that elect to review strictly local candidates, then the voting cabals could easily hold sway again, even with the excellent improvements above implemented. If majority voting is from outside the area, but the local reviewers collude to reject, it will take far fewer worldwide reject votes to reject a candidate, will it not?
I'd be worried that reducing the amount of local reviewers would just force botnets and cabals to expand their reach. If they're able to fully hijack the reviewing process for an area the size of Germany then I would imagine that there's nothing in particular limiting their actions and if forced to do so they could take over much larger regions. Reducing the impact local reviewers have would just cause them to no longer be local threats.
I have more faith in the people making botnets being able to extend their network to cover a greatly expanded region than I have faith in Niantic's ability to neuter them. The people subverting the wayfarer system are better staffed, motivated, and financed than the skeleton crew actually running wayfarer and that basic power imbalance needs to be addressed before moving against them.
@Duiomar-PGO I get where you're coming from, and that's a legitimate concern. I thought about that but I think their problem doesn't scale. It's not all of Germany, I think, just one region. If the botnets do manage to scale it then Niantic has a worldwide problem rather than a regional one and they are more likely to address it.
It definitely doesn't scale for human reviewers colluding.
OTOH, Niantic has an overall weak track record against systemic abusers.
Certainly an improvement such as the first idea is needed.
i think the idea of selecting a category and redirecting if it is a non- starter. A key point for me is that the categories need to be ore intuitive - possibly it’s the language but I struggle with the current what is it to find a good fit. But that should be straightforward to work on. I like the idea that this would help the submitter and the reviewer experience.
Both the other 2 are points need addressing but I’m not sure this is best way.
As a submitter in a 2 years plus area I do want something better. But not sure how it would work as the chance is the expanding range becomes self defeating. Eg if there are 2 slow areas nearby there will come a point quite quickly when the number being pushed into the wider pool along with those in the pool anyway causes stagnation.
I think it’s a starting point but it needs some serious number crunching by Niantic who are the only ones with details of the program that was written and the numbers of submissions and active reviewers.
I’m not convinced that having non-local reviewers would provide a solution to situations where coordination is taking place to manipulate the results. It seems to be operating in defined areas. I don’t see the need for a universal change to tackle a limited issue. What is needed is a way of picking this up and acting on it promptly - resources in the form of staff.
I think your last sentence hits the nail on the head. The first change the wayfarer team needs to make is getting niantic to expand their payroll, then they can start to make meaningful updates. The team now is hardly able to keep things from falling apart much less make improvements, they need more resources to work with.
@Elijustrying-ING I get what you're saying but staff has two problems-- one is that it doesn't scale well. The other, which we've seen from appeals among other things, is that staff make mistakes too. It's generally better to build systems that can prevent as many problems as possible rather than throwing heads at cleaning up the issues after they occur.
Addressing it with staff also requires that the problem be noticed.
I 'd tweak "What is it category" to be Which Criteria are you nominating it as "Explore, Gather, Exercise" That puts the purpose of a submission as the first thing a submitter sees
I'd put a checkbox before the final submit button that says "I know the ineligible categories like K-12 Schools and hearby testify this nomination doesn't fit ineligible category"
The potential problem I can see with number 3 is, in an ideal world where there's no cheating, local knowledge is needed a lot more for things like resteraunts/cafes or even just things under construction
@Gazzas89-PGO Is local knowledge really available under the current system? I'm not so sure it is.
My local play area includes stuff 150+ km away, I get upgraded stuff from all over the United States, and my home location is on another continent. I've encountered plenty of under-construction stuff and restaurants/cafes/pubs in all three ranges and the primary driver of my ability to review it is the quality of the supporting information.
I've certainly been able to spot and document fakes 150km away and 12,000 km away.
@Hosette-ING good point about more staff creating a larger quality control issue.
I agree if you create good systems/structure then it is a more solid foundation.
However I do think that there needs to be more people ( I get feeling that currently there are extremely few) with an investigative role and real teeth to tackle what they find.
i would regard my “local area” being the U.K. I see things in here from nearby European countries that I’m not familiar with, and although translation may be relatively quick it all adds time etc to assessing. I frequently do searches to check on the validity of a submission and I don’t think I would find or very easy. It is all quite off putting.
Also it is easier to spot some repeat problems if you are local - a wider pool would see these less so not pick up on the issue.
@Elijustrying-ING I'm not suggesting that there shouldn't be local reviewers. On the contrary... local reviews are valuable. I'm just arguing that they shouldn't have a majority vote on local candidates because that creates an abuse vector, and that abuse vector is being exploited in many locations.
I'm in the San Francisco bay area so my local review area extends over a large swath of California. My upgraded reviews come from all over the United States and occasionally Mexico or some random minor outlying island.
Technically that's what it's meant to do, and I'm not thinking about locals in numbers, I believe they should they should home/play locations get stuff nearby and rhat those votes are considered more for votes than say someone 100km away (as well as their ratings count too etc.) I may be misremembering that though.
Either way, locals are more likely to low that something is true (say a resteraunt/bar is the focal point of a small town) than people at the very distant reaches are
I think @Elijustrying-ING makes a really good point if you're thinking globally and not just the US case. In the US, within 100km (or 500km or more) the language is almost always English and the wayspot candidates will generally be similar things. However, for many locations, expanding to give non-local reviewers a majority vote would end up with candidates that need reviewers to understand supporting info at an even greater disadvantage.
Reviewers won't want to, or be able to in cases where Google Translate fails, read through articles about restaurants or proving importance of notable people. They won't know things like is a "Grade II listed historical building" common or not, or that in some neighboring country the local convenience store is the same as a local pub because everyone in town gathers there for a drink outside on the weekends.
This is already an issue in those locations, it pops up in the forums pretty often.