Waiting Times and the Distribution of Reviews .... dead cells

Raachermannl-INGRaachermannl-ING Posts: 1,070 ✭✭✭✭✭

Since lots of threads in the general section got lost, I will renew this thread...

Part 1: Analysis of the Distribution of the shown review candidates

The following maps show a heatmap of my last ~1500 reviews and a map with a S2 level 6 grid and the marking of my location. I havent set a home nor a bonus location.

So you should see the following:

Most of the reviews come from the own level 6 cell and the 8 surrounding ones.

Comparing the different zoom levels one should see that inside the marked 9 cells is a higher density of reviewed candidates than in the surrounding area. The only clusters outside are around bigger cities.

A second thing to observe here is, that all the candidates outside of this 9-cell-box are from Germany, or at least very close to the border. So evry candidate from Austria, Switzerland, France, Belgium, Luxembourg, Netherlands or Denmark were not deeper than ~10km away from Germany. So there seems to be a cell based geofence, that blocks long range submission to be from foreign countries. I would assume cell level 9 for that algorithm, because of observations discussed later.

While the rule of thumb with the 9 cells is very old, the geofence is comparatively new. The rule of thumb existed already during OPR times (reviewing platform before wayfarer) before Upgrades were released. With the release of the Upgrades in early 2019 came lots of long range reviews. The farthest I can remember was something from the border between Poland and Belarus, also I remember stuff from Warsaw or Brno, which are now out of range. So the geofence should have been introduced together with wayfarer or only few months later.

Another thing to mention is, that its very likely, that most of the long range reviews (outside the box) are upgraded ones. The observations to proof this also go back to OPR, but already while PoGo was allowed to submit. What I saw back then was, that during a longer reviewing session the candidates got more regional at some point. I recognized this because I had to review my own PoGo submissions from regions with fast timescales. They appeared more frequently after a few hundred straight reviews. So I interprete this behaviour as "crawling through the updates". Since PoGo players are allowed to review, I didnt notice this again. I think because there are way more upgrades in the system now, and Nia changed something in the prioritization of Upgrades during February 2020.


Part 2: Waiting times and Dead Cells

The following picture shows how the reviewing times are in the regions where I and few close friends submitted lots of wayspots, the grid is Level 9 (~20km edge) and level 6 (~150km) :

  • green cells: few days .... few weeks
  • yellow cells: few weeks .... few months
  • red cells: few months .... ???

The orange marked region is kind of special. Usually it behaves like the red marked regions with disastrous waiting times, but during few events it woke up and behaved like the green area for few weeks. Those events were the activity waves during Upgrade release, during wayfarer release and partially during the first weeks of the 1st Covid-lockdown (march 2020).

So one can see, that level 6 and level 9 cells have an impact. The impact of level 6 is a direct consequence of its role in the distribution algorithms from above and the picture shows this too, but it can differ on level 9 scale more than in my example region. So for example in Rhine-Ruhr-metropolitan area (Dortmund, Cologne and other cities there) or in the region around Frankfurt, there are higher impacts of the level 9 cells than in my region. But also for my level 6 home cell (with the orange and red marking) there are differences: not only the orange region changed its behaviour, also the whole north of the level 6 cell isn't as dead as the south. Afaik the town Dessau-Roßlau has timescales of a few weeks.

So there is the assumption for further prioritization:

The algorithm compares the absolute number of wayspots in a level 9 cell to those of its neighbour cells and prioritizes the region with less numbers.

One big hint for this is, that costal regions are very prioritized. In the water are no wayspots, so this helps those cells a lot. Only outliers there are costal cities like Hamburg, that are huge enough to cause a drop in their surroundings. In my region it's a similar situation - the orange/red region has 4 bigger cities lined up in its south: Jena (115k inhabitants), Gera (95k), Zwickau (95k) and Chemnitz (250k). Besides this their surroundings are also dense settled, because of a mining history in those regions. In comparison to this the green cell south of this region has no cities of that sizes, there are more mountain ranges and forests - so way less population and way less total wayspots leads to way more prioritization by the algorithm.


Part 3: Small scale side-effects on waiting times

A candidate, that has wayfarer status in voting, blocks candidates in neighbour level 17 cells to switch from in queue to in voting, except for upgraded ones.

Looking at my own submissions this is totally true. In my home village I submit lots of candidates in neighboured cells, and I puzzled over the ridiculous switching between in queue and in voting for a long time. Some candidates switched instantly to voting, some needed lots of months for this. So few days ago someone dropped this theory, and it fits all my pending submissions states.

«1

Comments

  • SeaprincessHNB-PGOSeaprincessHNB-PGO Posts: 1,609 Ambassador
    edited January 2021

    I haven't fully processed everything in here but this is a great analysis. My concern is how do we fix the problems that this has uncovered? Niantic has to be aware that this is how it works. Do they care about the impact this has on the health of the Wayfarer process? I've seen so many people give up reviewing because they've run into the problems you outline about the red cells. If nothing of ours gets through the system without an upgrade, the effort is not worth the reward. This is especially true when the system has been flooded with new reviewers who don't know the rules well. We put all the effort into getting that ONE UPGRADE only to have it land in front of reviewers who don't understand that giving it a 1 or 2 in any section will cause it to be denied (when it is a perfectly fine submission).

    I feel like they implemented this programming to help rural areas and that's a fine goal. But it shouldn't be all or nothing. It should be 60/40 so that all areas still have a chance to get approved POI without an insane amount of effort.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,388 ✭✭✭✭✭

    I love when anyone takes the effort to collect and analyze the data in this way. It's too late for me and my mind is already tired, but really: congrats for all the effort that you've put into this post.

  • AScarletSabre-PGOAScarletSabre-PGO Posts: 754 ✭✭✭✭✭

    Your post post got me thinking @Raachermannl-ING. I was speaking to somebody on the London Discord the other day about Wayfarer. Two S6 cells cover London area (that is to say, London postal addresses). I was informed anything within E Postcode area is a "dead area" and supposedly not because of a "lack of reviewers". The "East London S6 Cell" (as I'm going to arbitrarily call it) does cover a large area of costline. I'm told there's still large numbers of "scanned redacted" nominations in the queue all over London

    This cell ends at the Greenwhich Meridian, so it misses all of the main tourist areas, besides arguably, Greenwich itself (which would include the Millennium Dome, but various museums in Greenwich are west of the line). The cell line is a little to the east of the QE2 Olympic Park in Stratford. Indeed, Stratford High Street Station on the Docklands Light Railway is right on the line.

    The cell where I live is to the south of the "East London S6 Cell" and has even more sea in it than that cell and includes a tiny part of France. I'm shocked if I have to wait more than a week for decisions on my nominations. I can confirm that I did make one nomination in the "East London S6 Cell" (with a Dartford postcode) and it went into voting in October and has been stuck in voting ever since.

  • AScarletSabre-PGOAScarletSabre-PGO Posts: 754 ✭✭✭✭✭

    You know, what? Since I live very close to the boundary between two S6 cells, I could nominate two Wayspots right on the boundary, one either side, and see what happens. Will be a fun little experiment.

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    yes, i live at the border of 2 S2-Lvl6 Cells also.

    and also a cross where 4 S2-Lvl9 Cells meet. I can confirm what you are telling us.

    on the one side, i can understand niantic: they want the submissions in those lvl9-cells to be tracked faster, when there are not many in these lvl9-cells. chances are high that your submission will go much much faster. i tested it in a town that is split by these lvl9-cells, and indeed, the one side of town was fast, the other side is still in voting since June.

    It is nice that you went thru all the effort, but this has been reported alot here, and people are in the know. Surely Niantic wouldnt comment on that, but it still shows how broken the system is.

  • AScarletSabre-PGOAScarletSabre-PGO Posts: 754 ✭✭✭✭✭
    edited January 2021

    Further to my previous posts in this topic, it is interesting to note that S17 cells do not align with S6 cells. I've identified a potentially valid submission* that straddles the border between two S6 cells and a valid position on which to nominate this submission which would be on the S17 cell that falls under two S6 cells.

    *A hotel and day spa - but failing that there is a nature reserve next to it with more potential Wayspot potential.

  • Rostwold-INGRostwold-ING Posts: 172 ✭✭✭✭
    edited January 2021

    That sounds like a mapping error, the whole reason the cell system is useful is that doesn't misalign like that, allowing for increased precision through extra digits.

  • SeaprincessHNB-PGOSeaprincessHNB-PGO Posts: 1,609 Ambassador

    @NianticCasey-ING We deserve this to be addressed. Those of us who live inside these dead cells are some of the most dedicated, hard-working reviewers. We have data that proves our efforts are being spent on others with almost none on our own efforts to add POI for ourselves. This is not fair or equitable. I don't mind assisting rural areas or places that need help. But it can't and shouldn't be this lopsided. We must be able to get POI in our own areas in front of reviewers without an upgrade.

  • WheelTrekker-INGWheelTrekker-ING Posts: 3,388 ✭✭✭✭✭

    I don't know what you're using to check that, but it's either wrong or you haven't zoomed in enough to notice it properly. Each L6 contains four L7 cells, each L7 cell contains four L8 cells and so on.

  • Raachermannl-INGRaachermannl-ING Posts: 1,070 ✭✭✭✭✭

    No, that's not true. It's like @WheelTrekker-ING wrote.

    If you found something, that doesnt fit the grid, then possibly you had a bad plugin. For example the plugin, that creates the bright green level 17 grid in the maps of the reviewing interface is very inaccurate. I think its wayfarer+, that does this, but not sure. It has few metres offset....

    Since Niantic introduced, that clicking on the texts of the nomination opens it as google search, I see no reason to use any plugins. Darkmode and one-click-translation was the only reason for me to use plugins at this point. Now I only use the parts, that create the heatmap....

  • AScarletSabre-PGOAScarletSabre-PGO Posts: 754 ✭✭✭✭✭

    It was not Wayfarer+ but another plugin and also another website I tried. I was surprised to. I am not sure what's going on.

  • Raachermannl-INGRaachermannl-ING Posts: 1,070 ✭✭✭✭✭
    edited January 2021

    For IITC this is my favorite cell plugin set: https://github.com/Wintervorst/iitc

    It's totally accurate according to how Niantic uses the cells. Possibly yur other sources are accurate, but Niantic is inaccurate, and the S2 plugin provider needs to fit Niantics inaccuracy .... who knows xD

  • Hosette-INGHosette-ING Posts: 3,470 ✭✭✭✭✭

    @Raachermannl-ING Thank you for this absolutely excellent research. I read it a few times to digest it all, and I think it's clear.

    @AScarletSabre-PGO Your comment about the "east London" cell is an interesting. I live in California, but my bonus location is in central London. I have many reviews in the "west London" cell as well as the "east London" one. Also the two to the south of those. I have nothing from the cells to the north and west of the "west London" cell. My reviews also stop just west of Colchester and plotting them against an S2 map seem to stop at the L9 boundary that is just east of Sudbury.

  • sogNinjaman-INGsogNinjaman-ING Posts: 3,313 ✭✭✭✭✭

    I have to ask, if you live in California why are you reviewing things for London, it's not exactly close to you?

  • Raachermannl-INGRaachermannl-ING Posts: 1,070 ✭✭✭✭✭

    I cant say anything concerning bonus location from own expieriences. I have set neither bonus nor home location. For bonus location stuff @BitKloepplerin-ING could tell more. She has her bonus location set on Malta and told me her guesses few weeks ago, but I cant find it ..... what I know out of mind is, that she gets stuff from Malta, and the south of Sicily. I guess her guess was, that the bonus location works with 1+8 level 7 cells. I'll ask her again .... this question can easily be solved with that example, if there were submissions from Lampedusa too, or not.

  • Hosette-INGHosette-ING Posts: 3,470 ✭✭✭✭✭

    @sogNinjaman-ING I visit London once or twice a year (except in 2020, dammit), and I hacked my first portal in Trafalgar Square. I love the city and know it fairly well. Why wouldn't I review things in the area?

    @Raachermannl-ING If you're interested in looking at my data I can send it to you. I'll reach out to you.

  • SPD85-PGOSPD85-PGO Posts: 170 ✭✭✭✭

    In addition to the direct problem of the dead cells, this practice slows down the entire Wayfarer system. I know many people who stopped reviewing because they don’t get anything local and are stuck reviewing things in vacation areas, other cities, and college campuses 100 miles away. Even I review less than if I received local nominations. This means fewer reviews, and longer wait times, which causes more people to slow or stop their reviewing.

  • Xmacke7x-INGXmacke7x-ING Posts: 220 ✭✭✭✭

    In priority areas their is no incentive to review and in dead the motivation to review gets lost. It is a slippery ****

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    my sons told me they never get anything I submitted, and I seldom get anything that someone submitted here. I do understand that niantic wants to avoid that people vote biased.

    That perverts the whole action of setting a home location. They just went from that system to Germany-Wide + France + Belgium + Austria + Switzerland + Netherlands -+ Poland + Czech without letting people know.

    I have never ever read that there is some clarification why Niantic changed the system

  • SeaprincessHNB-PGOSeaprincessHNB-PGO Posts: 1,609 Ambassador

    This is what I came to add to this post. I will admit that I stopped doing reviews for a while because I have been focused on other parts of PGO. But the last few days, I have really tried to do a lot of reviews. In the past, I would gain a little bit of an agreement percentage during the time I reviewed and then several hours later, I'd log back in to see that my agreement % had made major gains. That doesn't seem to be happening right now. I'm gaining some but not at the same rate that I have in the past after doing long review sessions. And I know it's not just because I am not agreeing with other reviewers. My rating has been in the highest category (currently Great) for all but maybe 3 days since Wayfarer went live (once I really got going on the system - 7450+ reviews completed).

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    yes, Seaprincess, that is because the reviews are not fulfilled. there are sooo many Submissions floating in the system, only Niantic knows.

    These % will only slowly come in into the next few weeks.

    If Niantic would remove the Cork, i think we would get many many things accepted and rejected, and also our rating would climb fast.

    We had this last Time in March when C-19 happened. Somehow Niantic pulled the Cork, and we got alot of %

  • HaramDingo-INGHaramDingo-ING Posts: 1,725 ✭✭✭✭✭

    Reporting based on my area of Sydney:

    Part 1: Distribution of the shown review candidates

    Taken from Wayfarer+, you're right. It does take it from my reviewing cell plus the eight around. I think there is some sort of luck that there are two absolutely empty ocean cells I don't have to get reviews from. I will assume anything else reviewed outside of these nine central cells are all upgrades or from the bonus location (which at the moment is Tonga...), which makes me wonder how many reviews someone will do in Outback Australia.

    Part 2: Waiting Times and Dead Cells

    While in my post I only gave a radial example of average times of nominations, I've split the cells in the Sydney Metropolitan area in accordance to time in review:

    Because the only real split is between the Hawkesbury, upper Hornsby and north Northern Beaches LGAs, the data is as expected. Again, it is strange that there is a slightly lesser average time in Parramatta (30) than there would be in Blacktown (35).

    Notice The Oaks at the bottom-left, we had data of 7 different nominations across the town and they came through in 1, 1, 1, 2, 2, 2, 4 days. I am probably going to post the diagram above in mine as well.

    Part 3 is also an observation that I have seen as well (very small cell levels being blocked from transition from In Queue when something else in it is also in voting). I think Wayfarer attracts quite a lot of data-hungry people and data analysts and I think Niantic should capitalise on it.

  • oscarc1-INGoscarc1-ING Posts: 366 ✭✭✭✭✭

    Awesome info! What timeframe did you look at for those averages? Was it different in say, March, as compared to December 2020?

  • Ochemist-INGOchemist-ING Posts: 355 ✭✭✭✭✭

    I intentionally did this exact experiment in October. I submitted almost identical information kiosks at trailheads at different entrances to the same large park in a very rural area. The submissions were probably made within 15 minutes of of each other. They were separated by 1.2 km as the crow flies, but submission A was in very, very slow L6 cell and submission B was in a reasonable one. (I should note that cell A cannot possibly be slow for lack of reviewers as it contains most of the Washington D.C. metropolitan area.)

    Submission B in the faster L6 cell went into voting in less than 48 hours and I received a response about 10 hours later.

    Submission A in the unacceptably slow L6 cell (nothing moves at all ever without upgrades in other parts of it) went into voting in probably 48-72 hours, but it took 72 days to get a reply.

    This isn't helping rural areas if they have the misfortune of being in slow L6 cells and is truly counterproductive.

  • HaramDingo-INGHaramDingo-ING Posts: 1,725 ✭✭✭✭✭

    Thanks Oscar! Timeframe is for the entirety of 2020 with removed outliers during the April-May review blitz and where nominations were queued back due to another nomination being in voting in the same or nearby cell. Mount Tomah nominations were done throughout the year, the Oaks was done over two months, Kurrajong was done in May and July, and the Northern Beaches throughout September to December. So I had enough samples in different months in different areas to make a conclusion.

    The differences are far more pronounced in the orange and red zones, where reviewing times really have increased over time as the year progressed (especially between June to September). For green zones, they turned out pretty much the same.

  • HaramDingo-INGHaramDingo-ING Posts: 1,725 ✭✭✭✭✭
    edited January 2021

    .

    Post edited by HaramDingo-ING on
  • WheelTrekker-INGWheelTrekker-ING Posts: 3,388 ✭✭✭✭✭
  • Raachermannl-INGRaachermannl-ING Posts: 1,070 ✭✭✭✭✭

    Revisiting my graphic from the initial post, I have a few observations from the last few months:

    We had a lot of nominations in these two blue marked cells.

    In the eastern one I'm submitting a lot since January. During January the reviewing time was between 2 and 5 days there. Now there is 3 weeks and more needed. From my set of 14 submissions 3 weeks ago only one single example had a result until now.

    In the western cell a friend of mine is an active submitter. Afaik the timescale there is also growing a bit, but not as hard as in my cell.

    Further: one of our measures is another friend and active reviewer, who usually posts jokes, if she gets lots of our submissions. She recognized, that she doesnt get my cell as frequently as the other one now, although I consequently give her replenishment submissions evry 2 weeks....


    So I'm curious about whether we see here how a level 9 priority cell loses it's priority, or maybe it's only a general trend right now, that evrything gets slower .....

    Maybe @HaramDingo-ING has latest numbers from Australia?

  • Rodensteiner-PGORodensteiner-PGO Posts: 1,705 ✭✭✭✭✭

    we are also submitting heavily in a lvl9 cell that basicly had nuffin. the amount of wayspots is growing rapidly, and we all agreed that after roughly 2 months the priotity is still there, albeit it is a bit slower now (from 5 days to 14 days).

    I guess there are different priority tiers.

Sign In or Register to comment.