Unlimited quiz attempts is going to lead to a lot more upgrades of ineligible wayspots and a lot more approval of stuff that is categorically ineligible as well as disagreements for those of us that actually review according to the guidelines.
That's a good idea. A non-harsh penalty for people who may have just been confused about one thing, but a harsher one for people who clearly need to take the time to better understand criteria.
Let's say it's still good that you can take an unlimited number of tests.
But to be able to do so after 15 minutes is unwise.
Also, without a mechanism to solemnly dismiss players who can only do low-quality reviews after passing the test, it would be foolish.
If we only implement a system that allows unlimited attempts, there is a very good chance that we will have an influx of poor quality reviewers, as many people here have posted.
That's how easy this test is. This test is that easy, because it is a test where the answer is written in the description of the question and the choices.
It's even easier than the test conducted by Operation Portal Recon, which preceded Wayfarer.
If you can understand the meaning of the words, you can probably pass the test even in elementary school.
First of all, the level of this test needs to be at least as high as OPR, or even higher.
What we are looking for is not poor quality reviewers, but quality reviewers. We want good quality reviewers.
Also, you may not want to admit it, but a large percentage of Pokémon GO trainers are multi-account.
This is of course illegal according to your game's terms of service.
But you are not taking any action.
And it is also clear from this forum that there is no end to the abusive behavior that is currently increasing the number of low-quality Wayspots in certain games.
The accounts of these players double and triple in size.
If you want to destroy the LightShip database so that it is not worth making it unavailable again for other companies to compete with Niantic, then I can understand this attempt.
Unless there is a mechanism in place to immediately remove the nominators who can only submit low quality POIs, and the judges who can only review low quality POIs, I find this to be foolish and will oppose it.
Please take these measures and then submit the draft again.
So for example, in college classes with those "dynamic study modules"
Sure, I could study, read the book chapters, then take the "quiz"
Oooorrrr I can jump straight into the quiz since it has no penalties (and if it tells me what was wrong) I can keep taking it over and over until I pass and learn nothing, but still get a passing grade.
I'll add my voice to the chorus of people who think that unlimited quiz attempts with a very short retry period is a bad idea UNLESS the database of questions is quite large and the user gets a random subset each time. Even so, this becomes a monkeys and typewriters problem-- the player will eventually get lucky and pass the test by sheer chance.
If you want to make retries easier make the time sort of exponential. One retry after 15 minutes, one after an hour, one after four hours, one after one day, one after one week... Doing it this way means someone who basically knows the criteria but barely missed can probably pass on the second try, but someone who doesn't know can't just keep hammering on it until they get lucky.
You could crowdsource test questions to get a larger pool. Allow people to submit questions and then Niantic can curate the submissions to build a larger database to choose from.
Unlimited tests, so your saying you don't care about quality we just want more more more.
Is there any point in trying to pretend you actually care about the quality of the POIs or the criteria, because your actions and words speak otherwise.
Everyone understands implicitly that NIA has not erased most of the POIs in consideration of Pokemon and others. However, I think the NIA should learn from its history that most companies that ignore quality for the sake of profit will fall as a result of poor customer satisfaction. I am thinking that maybe NIA is planning to sell the whole system to another company someday.
Dear Niantic, please don't take 2 small steps forward and 1 giant step back. The other 2 changes are fantastic, but please please please for the sake of reviewer integrity, if you really want to allow unlimited attempts, at least make it more than 15 minutes. Between first and second attempts, okay make it 15 minutes if you must, but it should definitely increase in time intervals after that. I'd say a month at least, to a year after that.
If you only have a limited pool of questions, it's not hard to pass eventually given enough attempts. I know this from experience because that's how I [eventually] passed some mandatory policy quiz at work without understanding most of the questions...
This is great! Glad to have more reviewers in the process even if it took them a bit longer to learn the criteria. No chance they will be any worse than the current ones (present company excluded).
Getting the Niantic badge next to a nomination that's being reviewed by Niantic is good (definitely more preferable compared to it vanishing until it reaches a decision).
Focusing on only the top 2 rejection reasons seems OK to me. Of course, that assumes that reviewers are being competent when selecting those reasons.
I do echo the sentiments of everyone else about the quiz though. I don't mind multiple more attempts, but every 15 minutes just means it can easily be brute forced with no learning. I think I like @aleprj-PGO 's suggestion of gradually increasing the timescale between retaking the test.
So now we can see that Niantic doesn't care AT ALL who is going to review POIs. They just need more unpaid labor, no matter if done good or poorly. You failed the test 20 times and passed it at 21th by luck? Welcome!
I've made a basic version of such a quiz in the past with at least 3x the number of questions and a wider variety of coverage than what Niantic currently tests for. If they truly will listen this is something I'd be happy to work with them on.
Personally, I think that we need to stock up on about 100 questions in the background of a single field of study and not waste time trying to analyze them.
And we should also attempt to update a certain percentage every six months.
We also want to raise the level of the problems themselves.
I certainly like the idea that opening up more retesting opportunities will be more inclusive. I don't remember taking the test, but I do remember several silly questions that didn't quite align with how reviewing actually works. I recall some complaints about language/translation issues, too.
Maybe 15 minutes is just long enough that somebody who doesn't belong reviewing will get bored/ lose interest waiting to retest. Or maybe it'll be just long enough to review the guideliness Google the answers.
I also wonder if somebody who passes the test on their first try will have the same reviewer rating as somebody who failed 2+ times. Perhaps people who fail multiple times should start out having less impact on reviews and/or more frequent cooldowns.
sorry but unlimited tries is just crazy - if someone fails 3-4 times those shouldnt be in the system if they cant pass a basic test the first time.
cant say i feel anything positive will come out of this, if anything lazy ppl will just keep submitting crappy poi and legit nice poi is in voting or stuck que forever...
With the 15-minute cool-down, what will be achieved is that the bad evaluators can pass the exam and can evaluate, with all the inconveniences that this entails.
This would be solved if there was an exam to request Points of Interest; If this exam is passed, you can request nominations and you can take the evaluation exam. If the proposal review is not passed, it cannot be asked for, nor can it be evaluated.
This topic is not only left there, "toxic" players are also given the opportunity to request up to a maximum of 40 requests, one per day. This is a lot of spam and mindless request abuse that crashes the review system.
I am ready to subscribe to almost every word of colleagues who speak negatively about the changes in the exam.
The further you go, the more the quality of reviewing declines (I've been doing this since OPR, so I can tell), and this proposal will open an even wider gate for those who don't understand the criteria. Let's be honest, for a thinking person even OPR-era test was simple enough, in WF it is even simpler and the fact that someone manages to fail it twice causes me personally nothing but boredom and facepalm.
Quantity is no substitute for quality, unless of course you're still worried about the quality of your POI network.
A good improvement to the system, in my opinion, would be to point out mistakes when you fail on your first attempt, so that the potential reviewer will account for their mistakes in the second.
But it shouldn't just be showing the questions where the wrong answer is given, with the correct answer indicated. The idea should look something like this: "Unfortunately, you made mistakes and could not pass the test. Perhaps you should learn more about the principles of evaluating the following situations: ...", and list the common points in which the novice made mistakes. Definition of PRP, temporalities of objects, safe pedestrian access, etc.
Wayfarer forums: “Please do something to stop terrible reviewers & submitters. Maybe you could have a submitter test? Maybe even existing submitters should take it? Though maybe allow people more than two chances to pass the review test.”
Niantic: “We heard you, unlimited test attempts to review and submit. Don’t pass? Just watch a quick YouTube video/half an episode of a TV show and come back and try again! Zero penalties for multiple failures!”
Comments
They could at least increase the time progressively with each attempt.
15min/4hours/1day/1week/2months and then it tops at 2 months or something like that.
Ah, and I forgot to say that the other 2 changes are great. Thanks for that!
Unlimited quiz attempts is going to lead to a lot more upgrades of ineligible wayspots and a lot more approval of stuff that is categorically ineligible as well as disagreements for those of us that actually review according to the guidelines.
That's a good idea. A non-harsh penalty for people who may have just been confused about one thing, but a harsher one for people who clearly need to take the time to better understand criteria.
@NianticTintino
Let's say it's still good that you can take an unlimited number of tests.
But to be able to do so after 15 minutes is unwise.
Also, without a mechanism to solemnly dismiss players who can only do low-quality reviews after passing the test, it would be foolish.
If we only implement a system that allows unlimited attempts, there is a very good chance that we will have an influx of poor quality reviewers, as many people here have posted.
That's how easy this test is. This test is that easy, because it is a test where the answer is written in the description of the question and the choices.
It's even easier than the test conducted by Operation Portal Recon, which preceded Wayfarer.
If you can understand the meaning of the words, you can probably pass the test even in elementary school.
First of all, the level of this test needs to be at least as high as OPR, or even higher.
What we are looking for is not poor quality reviewers, but quality reviewers. We want good quality reviewers.
Also, you may not want to admit it, but a large percentage of Pokémon GO trainers are multi-account.
This is of course illegal according to your game's terms of service.
But you are not taking any action.
And it is also clear from this forum that there is no end to the abusive behavior that is currently increasing the number of low-quality Wayspots in certain games.
The accounts of these players double and triple in size.
If you want to destroy the LightShip database so that it is not worth making it unavailable again for other companies to compete with Niantic, then I can understand this attempt.
Unless there is a mechanism in place to immediately remove the nominators who can only submit low quality POIs, and the judges who can only review low quality POIs, I find this to be foolish and will oppose it.
Please take these measures and then submit the draft again.
So for example, in college classes with those "dynamic study modules"
Sure, I could study, read the book chapters, then take the "quiz"
Oooorrrr I can jump straight into the quiz since it has no penalties (and if it tells me what was wrong) I can keep taking it over and over until I pass and learn nothing, but still get a passing grade.
Perfect.
I'll add my voice to the chorus of people who think that unlimited quiz attempts with a very short retry period is a bad idea UNLESS the database of questions is quite large and the user gets a random subset each time. Even so, this becomes a monkeys and typewriters problem-- the player will eventually get lucky and pass the test by sheer chance.
If you want to make retries easier make the time sort of exponential. One retry after 15 minutes, one after an hour, one after four hours, one after one day, one after one week... Doing it this way means someone who basically knows the criteria but barely missed can probably pass on the second try, but someone who doesn't know can't just keep hammering on it until they get lucky.
You could crowdsource test questions to get a larger pool. Allow people to submit questions and then Niantic can curate the submissions to build a larger database to choose from.
You spoke and we listened!
I hope you’re listening to all the users speaking in this thread that the 15 minute wait is a bad idea @NianticTintino
What in the world...
Did I say to let users who failed the test twice to experience the orientation and give them one last chance to retake the test?
Unlimited tests, so your saying you don't care about quality we just want more more more.
Is there any point in trying to pretend you actually care about the quality of the POIs or the criteria, because your actions and words speak otherwise.
Stupid idea
Everyone understands implicitly that NIA has not erased most of the POIs in consideration of Pokemon and others. However, I think the NIA should learn from its history that most companies that ignore quality for the sake of profit will fall as a result of poor customer satisfaction. I am thinking that maybe NIA is planning to sell the whole system to another company someday.
@NianticTintino @NianticGiffard @NianticGiffard
Dear Niantic, please don't take 2 small steps forward and 1 giant step back. The other 2 changes are fantastic, but please please please for the sake of reviewer integrity, if you really want to allow unlimited attempts, at least make it more than 15 minutes. Between first and second attempts, okay make it 15 minutes if you must, but it should definitely increase in time intervals after that. I'd say a month at least, to a year after that.
If you only have a limited pool of questions, it's not hard to pass eventually given enough attempts. I know this from experience because that's how I [eventually] passed some mandatory policy quiz at work without understanding most of the questions...
So you're developing a system that will allow the "how to pass the Wayfarer test" YouTube videos even more traction?
This is great! Glad to have more reviewers in the process even if it took them a bit longer to learn the criteria. No chance they will be any worse than the current ones (present company excluded).
Getting the Niantic badge next to a nomination that's being reviewed by Niantic is good (definitely more preferable compared to it vanishing until it reaches a decision).
Focusing on only the top 2 rejection reasons seems OK to me. Of course, that assumes that reviewers are being competent when selecting those reasons.
I do echo the sentiments of everyone else about the quiz though. I don't mind multiple more attempts, but every 15 minutes just means it can easily be brute forced with no learning. I think I like @aleprj-PGO 's suggestion of gradually increasing the timescale between retaking the test.
So now we can see that Niantic doesn't care AT ALL who is going to review POIs. They just need more unpaid labor, no matter if done good or poorly. You failed the test 20 times and passed it at 21th by luck? Welcome!
I've made a basic version of such a quiz in the past with at least 3x the number of questions and a wider variety of coverage than what Niantic currently tests for. If they truly will listen this is something I'd be happy to work with them on.
@NianticTintino
One quick question, will nominations reviewed by Niantic prior to this update also show the icon? That would be really nice to see.
Personally, I think that we need to stock up on about 100 questions in the background of a single field of study and not waste time trying to analyze them.
And we should also attempt to update a certain percentage every six months.
We also want to raise the level of the problems themselves.
Unlimited quiz attempts is half a joke, 15 minutes between tests is ridiculous.
You could at least keep the 30 days between attempts. Or even better add 30 days to the waiting time per attempt.
Then people will do their research before taking the test instead of hoping they pass on one of their any attempts
I certainly like the idea that opening up more retesting opportunities will be more inclusive. I don't remember taking the test, but I do remember several silly questions that didn't quite align with how reviewing actually works. I recall some complaints about language/translation issues, too.
Maybe 15 minutes is just long enough that somebody who doesn't belong reviewing will get bored/ lose interest waiting to retest. Or maybe it'll be just long enough to
review the guidelinessGoogle the answers.I also wonder if somebody who passes the test on their first try will have the same reviewer rating as somebody who failed 2+ times. Perhaps people who fail multiple times should start out having less impact on reviews and/or more frequent cooldowns.
If someone fail several time a such easy test he shouldn't be able to do nomination
Start Wayfarer now but i cant see no different with 4.3 😪
Not out yet.
sorry but unlimited tries is just crazy - if someone fails 3-4 times those shouldnt be in the system if they cant pass a basic test the first time.
cant say i feel anything positive will come out of this, if anything lazy ppl will just keep submitting crappy poi and legit nice poi is in voting or stuck que forever...
With the 15-minute cool-down, what will be achieved is that the bad evaluators can pass the exam and can evaluate, with all the inconveniences that this entails.
This would be solved if there was an exam to request Points of Interest; If this exam is passed, you can request nominations and you can take the evaluation exam. If the proposal review is not passed, it cannot be asked for, nor can it be evaluated.
This topic is not only left there, "toxic" players are also given the opportunity to request up to a maximum of 40 requests, one per day. This is a lot of spam and mindless request abuse that crashes the review system.
The title is "Upcoming Wayfarer..."
The first line of the post reads: "...I’m very excited to share that we’ll be shipping some new updates with Wayfarer 4.3 next week..."
So this is an announcement. Please wait.
I am ready to subscribe to almost every word of colleagues who speak negatively about the changes in the exam.
The further you go, the more the quality of reviewing declines (I've been doing this since OPR, so I can tell), and this proposal will open an even wider gate for those who don't understand the criteria. Let's be honest, for a thinking person even OPR-era test was simple enough, in WF it is even simpler and the fact that someone manages to fail it twice causes me personally nothing but boredom and facepalm.
Quantity is no substitute for quality, unless of course you're still worried about the quality of your POI network.
A good improvement to the system, in my opinion, would be to point out mistakes when you fail on your first attempt, so that the potential reviewer will account for their mistakes in the second.
But it shouldn't just be showing the questions where the wrong answer is given, with the correct answer indicated. The idea should look something like this: "Unfortunately, you made mistakes and could not pass the test. Perhaps you should learn more about the principles of evaluating the following situations: ...", and list the common points in which the novice made mistakes. Definition of PRP, temporalities of objects, safe pedestrian access, etc.
Wayfarer forums: “Please do something to stop terrible reviewers & submitters. Maybe you could have a submitter test? Maybe even existing submitters should take it? Though maybe allow people more than two chances to pass the review test.”
Niantic: “We heard you, unlimited test attempts to review and submit. Don’t pass? Just watch a quick YouTube video/half an episode of a TV show and come back and try again! Zero penalties for multiple failures!”