Changing the 5 star system to a 4 star system

2»

Comments

  • GearGlider-INGGearGlider-ING Posts: 1,296 ✭✭✭✭✭

    The two fields that get discussed the most on this topic are historical/cultural significance and visual uniqueness. Neither of these things is a requirement for a wayspot, unlike safe pedestrian access.

    Considering that Wayspots have been rejected for the sole reason for not meeting the criteiria of "Visual Uniqueness" and "Historical/Cultural Significance" (as shown by the emails I've posted earlier) I'm going to have to say you're wrong on that one.

    You can say that they would not have been rejected for those reasons if the other sections got higher ratings, but then why would niantic not include the other reject reasons at all? And why would Niantic ask reviewers to rate these categories if it wasn't as much of a concern they met them or not? Telling people categories can be rated 1* can can still be approved when failing to meet these categories are feedback given reject reason just doesn't match up.

  • Hosette-INGHosette-ING Posts: 1,915 ✭✭✭✭✭

    @GearGlider-ING There's a difference between "Considering that Wayspots have been rejected for the sole reason for not meeting the criteiria..." and "The email notification tells us (that same thing)". Consider the number of complaints in this forum about how much Wayfarer does wrong. Think about all the cooldowns for things that don't make sense. Think about the bizarre rejection reasons, the amount of time that things get stuck in review or waiting to be reviewed, random ratings drops, things not being synced to games, the amount of abuse that gets through the system...

    I'm going to ask you to consider believing one of two things based on other evidence that you have about Wayfarer:

    • The rejection reasons provided in rejection email provide a 100% accurate and provide very precise information about why a candidate was rejected
    • Niantic doesn't always provide good information in rejection email

    Given how many other well-documented problems there are with Wayfarer isn't it at least plausible that rejection email doesn't give us a perfect picture of why something was rejected?

  • Roli112-PGORoli112-PGO Posts: 2,098 ✭✭✭✭✭

    I don't understand how someone could defend a plausibility with so much passion and zero evidence and gets upset when everyone disagrees. Your initially gave your opinion as if it was fact and have not provided any evidence except "Niantic makes mistakes" oh and a secret conversation (where have i heard that before?).

  • pkmnsearch2-PGOpkmnsearch2-PGO Posts: 249 ✭✭✭

    oh wow. thank you for this information.

    Since the start i never like the star rating. Why do i need to rate title/description or safe pedestrian acess if there is a reject reason for it? Why do i have a star rating for location accuracy when wayfarer says for me to answer only 1* , 3* or 5* ???

    When i got informed about the historic visually unique reject reasons appear in the email, thats when i started to use 3* as a base. If by any chance i review (or reviewed before knowing about the need of 3* as a base) nomination that deserves less than 3* in historic and /or visually unique, 100% of the time is a straight 1* with a strong reject reason.

    once again, @phi2458-PGO thank you for the information.Now i am sure, I absolutly hate the star rating.

    I can't wait for the revamp using yes no maybe answers AND with realistic questions according to the Acceptance criteria page (https://wayfarer.nianticlabs.com/criteria#acceptance).

    Do you consider a great place for...? exploration, social gather, exercise.

    Does the title/description violates the guidelines?

    Does it has safe pedestrian acess/is it easily tangibile?

    Is it permanent?

    Is it easily identifiable from it surroundings?

    Is the location pin in the in-real-life POI location?

  • pkmnsearch2-PGOpkmnsearch2-PGO Posts: 249 ✭✭✭

    There are even Ingress AMA suggesting that 2* or less is a rejection (since everybody is always @ niantic users)... Once again, i don't consider them facts. But gosh... i really cant wait for wayfarer to be done with the star rating.

  • TWVer-INGTWVer-ING Posts: 391 ✭✭✭✭

    Your examples don't make sense. What you are suggesting (rejection reasons in the e-mail are wrong) is a simple reporting bug that can easily be investigated and fixed. And you compare that to complaints about design choices and bad reviewing.

    Cooldowns are a design choice, and it is a choice that I personally am happy about. Bizarre rejection reasons are because of bad reviewers. Yes they exist, and there are probably far more of them than you can imagine. Things taking time is partly a design choice, and partly a bad submitter to reviewer ratio. Sometimes things get stuck. But those are usually rare occasions in small area's. But those usually get fixed relatively fast when they surface. Random rating drops again are bad reviewing. It was explained that this was caused by people voting the wrong way on pre-reviewed nominations. It still saddens me that they "lowered the ding" on those. Things not syncing are also rare occasions, that usually get fixed within a day or a couple of days, and one occurance where things didn't sync to HPWU that lasted a lot longer, but that got fixed eventually as well. The amount of abuse that gets through is also bad reviewing.

    These are all way more complex problems than a simple reporting bug that takes a single programmer less than a day to investigate and solve. Am I saying that your theory is impossible? No, but I find it very unlikely.

  • pkmnsearch2-PGOpkmnsearch2-PGO Posts: 249 ✭✭✭
    edited April 2021
  • GearGlider-INGGearGlider-ING Posts: 1,296 ✭✭✭✭✭
    edited April 2021

    You're question forces me to choose between a very strict polarized option that refutes your point, and a wide reaching open to interpretation option that supports your point. That's just manipulative arguing right there.

    You said:

    The two fields that get discussed the most on this topic are historical/cultural significance and visual uniqueness. Neither of these things is a requirement for a wayspot, unlike safe pedestrian access.

    However, not only does Niantic ask us to rate them next to other criteria required for a wayspot, but they offer lack of these features as a rejection reason as well. And if you're just going to ignore that and say the email rejection reasons aren't the REAL reason a wayspot was rejected by reviewers, then I don’t think you have a good grasp on how rejections work.

    Post edited by GearGlider-ING on
  • Hosette-INGHosette-ING Posts: 1,915 ✭✭✭✭✭

    @TWVer-ING I think cooldowns are a design choice with bad unintended consequences. (I also suspect there's a bug.)

    I think the algorithm for choosing email rejection reasons is a design choice with unintended consequences. I don't have an opinion on whether or not there's a bug.

  • phi2458-PGOphi2458-PGO Posts: 163 ✭✭✭

    @Hosette-ING I have seen that you put so much effort. But I’m still not sure about your kernel idea. What is the kernel that you are going to tell us?

    I tried to organize what you said and guessed what you want to tell us. Please let me know if that is not what you meant.

    I guessed what you were going to tell us is that when we answer the reviewing questions, the algorithms should not be our concern. In other words, when we answer the visual uniqueness question, we should focus on the question itself and don’t think about if the answer would cause acceptance or rejection.

    In my opinion, I agree that it is what the system should be in theory. However, it has never happened in practice because we are humans, not machines.

    We are not merely reviewers. We also play the role of submitters and players. All of us have different purposes of reviewing, such as making the game environment better and maintaining the quality of spots. Either way, the final goal is to decide whether the nominations should be accepted or not. As a result, we do care about the algorithms because we want to know whether the effects of our answers are the same as what we intended.

    Since there is some empirical evidence shows that the weight of some questions is more than we expected. It is reasonable that reviewers rate the nomination according to the evidence. In my opinion, Wayfarer is not a master-slave system. Reviewers may not follow the original design. Instead, it should be an evolutionary system. Since the algorithms shaped the behavior of reviewers, the behavior of reviewers should also be able to shape the algorithms.

    Humans are complex. That is why we need sociologists. : )

    Please let me know if there is any problem.

  • Hosette-INGHosette-ING Posts: 1,915 ✭✭✭✭✭

    @phi2458-PGO My main frustration is that largely seeing misinformation being parroted as truth. And yes, I'm really quite certain that "a 1-2* in any field is a rejection of the whole submissions" is misinformation. I saw this go from people speculating to it being spoken as gospel. I believe that we should review by giving our best assessments based on the guidance that Niantic has provided to us... confusing and convoluted as it can be. And yes, I grok why players want to understand the algorithms. (I don't grok the absolute adherence to certain ideas, though. I get the impression that if the entire Wayfarer development team came in tomorrow and said, "Nope, that's not how it works" that people would still believe it was.)

    Unfortunately, a lot of the cases of players adapting to the algorithms have been abusive. I'm thinking people misplacing pins and moving wayspots to manipulate cells, fake wayspots approved by people collaborating and multi-accounting to do so, and the giant clusters of misplaced wayspots in some cities, players banding together to set alternative rules for their areas, etc. (Just tonight someone was telling me that they'd had yet another trail marker rejected because the reviewer cabal in their area doesn't like trail markers.)

    I review because I enjoy the virtual tourism aspect of it and because I want to bring high-quality wayspots into Niantic's database. Unlike most players I don't do it for upgrades or to populate my area-- I live in a place that was an early Ingress hotspot so the area around me is thoroughly saturated, and I submit very little so I'm almost always capped on upgrades. I review to give something back to Niantic for almost seven and a half years of excellent entertainment.

    The behavior of reviewers has definitely shaped the algorithms and the functionality. For example, we can no longer move pins more than ten meters through the standard review system. There are anti-bot mechanisms in place that seem to hit honest human reviewers far too often.

  • TWVer-INGTWVer-ING Posts: 391 ✭✭✭✭

    If it is a design choice and not a bug, that means many nominations are getting rejected for low scores on these categories.

    We have seen evidence of these reasons showing up alone and combined with other reasons. To me, that points heavily towards: A low rating on either of these categories can tank the entire nomination.

  • AisforAndis-INGAisforAndis-ING Posts: 782 ✭✭✭✭✭

    Bingo.

    I think the biggest elephant in the room is that Niantic has acknowledged that the current review questions within wayfarer are flawed.

    Whether it makes sense or not, wayspots currently need to be scored well in these categories to get approved. Even though the questions need to be improved, it does not make these questions irrelevant for wayspot approval.

  • Gendgi-PGOGendgi-PGO Posts: 2,797 Ambassador

    I get the impression that if the entire Wayfarer development team came in tomorrow and said, "Nope, that's not how it works" that people would still believe it was.)

    You selectively believe cooldowns to be a bug despite clarifications over the last year from Niantic that they are not. You selectively believe it to only be Casey's opinion that fire hydrants obstruct emergency services, despite clarifications it is the opinion of the team.

    It's easy to pick and choose statements to confirm one's biases.

  • phi2458-PGOphi2458-PGO Posts: 163 ✭✭✭
    edited April 2021

    Since only Niantic knows the exact algorithms, we could only guess it based on the evidence that we observed. Therefore, we cannot be 100% certain how the rating system works. According to the email screenshots and some cases on this forum, I tend to believe that low ratings in the "visual uniqueness" and the "historical/cultural significance" category would cause the rejection.

    You said you are quite certain that it is misinformation. But honestly, your arguments are not very convincing.

    Let's put this question aside for now. There is another question. Assuming that you are correct, what do you think of the consequence of the misinformation?

    You might say that reviewers would rate higher than they used to be. And that leads to inaccurate results. However, the results of such questions are never accurate because they are subjective questions, and Niantic didn’t set the baseline. In other words, when we answer the "visual uniqueness" and the "historical/cultural significance" question, Niantic didn’t tell us the standard of 1-star, 3-star, and 5-star. The results totally depend on the personal feelings of reviewers. In my opinion, the change of personal standards is just a minor issue to the system.

    Compare to other types of abuse that you mentioned, this is not a big problem, to be honest.

    Post edited by phi2458-PGO on
Sign In or Register to comment.