Story Rankings and Trending Section Changes Have Been Rolled Back


#1

Hello again Episodians,

What a crazy few days it’s been! As of 4pm PT yesterday afternoon we rolled back the changes to the trending section, returning to our previous frequency of running the rankings update every 4 hours. Note that it may take up to 48 hours for the trending section and your rankings to level out, and we appreciate your patience as we also continue to monitor the results on our end.

While this was going on, many of you asked to know more about how the rankings work so here is the general philosophy behind the algorithm. As a reminder the trending section is a reflection of the stories that readers in the app are spending their time in.

  1. A story’s ranking should reflect the volume of reads its achieved. Every completed read of a chapter counts towards your story’s rank, and you only start to accumulate reads once you publish your story.

With our latest update, returning the rankings update window from 24 hours down to 4 hours means that we’re calculating rankings based on the amount of reads your stories have accumulated within the last 4 hours (instead of within the last 24 hours).

In general, shorter update windows allow “hot” stories to more quickly climb to the top of the ranks in a short period of time, and favor virality; on the other hand, longer windows allow “consistent” stories that tend to get the same amount of reads over a long period of time to rise to the top.

  1. A story’s ranking should primarily reflect current reader behavior. A read that you get now should be more valuable than a read that you got, say, 6 months ago, in order for the trending section to actually highlight trending stories.

The faster that we reduce the value of a read over time, the more even the playing field for brand new stories; the slower, the more of an advantage existing stories have.

As an example, let’s consider a case between 2 stories:

Story A has been live for 6 months, and has accumulated 10,000 reads in that time.

Story B has just been published, and is starting with 0 reads.

If we never made the initial 10,000 reads less valuable over time, then Story B is actually starting at a 10,000-read deficit compared to Story A – meaning that if both stories were to get 100 reads in the next 4 hours, Story A would be placed much higher in the rankings than Story B even though they are equally popular amongst readers in that time.

To avoid the history bias, rankings are weighed more heavily towards the newest reads that a story’s achieved in the most recent ranking window. That way, we won’t discount the total amount of reads altogether, but the longer it’s been since a reader completed a chapter in your story, the less of an impact that read has on your current ranking.

Between these 2 factors, and when we looked back on our historical rankings data, we’ve noticed that in reality, story rankings don’t really change all that much within 24 hours. That means that we’re using up valuable server time and services to calculate #s that, 5 times out of 6 in a given day, aren’t really any different from each other. This is why we wanted to reduced the rankings calculations down to once a day; by cutting the redundancy out of our system, we could put our money and resources towards other areas of the platform.

However, when we made the shift to once a day update last week, we also inadvertently exposed some legacy issues (read: old code) with how we set up the system all the way back in 2014. Despite numerous tests, our changes caused some of the jobs that calculate ranks to run out of sync. This is why your ranks shifted so dramatically in a short period of time, and we’re sorry for the anxiety that’s caused everyone over the weekend.

In 2019, we still want to move forward with a system that actually runs efficiently and keeps all the rankings in sync. For now, we will leaves things the way they were through the holidays, and wish everyone season’s greetings and a happy new year once you get there.

With love,

The Episode Team


Update to Story Ranking and Trending Section Calculations
#2

I’ve seen this as well, that’s why I want a new ranking system. It really doesn’t change so there’s no need for servers to be updated 6 times a day.

But how would “calculating” rank for newly published stories work? Did I miss something?


#3

It will still say “calculating” until the next rankings calculation runs. Which could be 0-4 hours after its published depending on where the story falls in the window.


#4

ooof finally its fixed, the past few days have been a mess god bless


#5

So as @Arrows.Episode said in the last thread, this is going to make it harder for newer authors to break into the top positions, as they will need a consistent number of reads over 24 hours rather than a short spike to break through?

I.e.
Story A gets 2,000 reads the first hour after publishing, then 2,500 over the next 23 hours =4.5K total
Story B gets 200 reads an hour over 24 hours = 4.8K total

So story B will trend the highest as it is gets more reads over the 24 hour period.


#6

Thanks Episode Team x I know you guys are working hard to come up with the best way for story ranking. You didn’t mean for all this kerfuffle to happen so best of luck for future updates.


#7

Not quite. The rankings calculations run every 4 hours, so a quick spike would allow for a story to “break through” as you put it.

If we were to go back to the 24 hour window, both stories would still likely end up on the trending section because the difference between 4.5k reads and 4.8k reads is not large enough for one to really outpace the other.


#8

Thanks for clarifying, I’m glad they’re getting back to normal. Happy holidays to the team.


#9

Thank you for this Liz!

I only have one question:

Is this a new thing or is it how it has always been? Just asking because I’ve always thought that the ranks are calculated only by the reads a story receives between two ranking windows. Anyway this sounds pretty fair, I’m just curious :slightly_smiling_face:


#10

Its not new.


#11

Thanks so much for the explanation Liz :slight_smile:


#12

We all appreciate the time you’ve taken out to explained the situation to us!


#13

4 hours seems like a good solution and still gives new stories and smaller authors to work their way to the top while meeting your end goal. Thanks for listening to us, and I hope the new system you guys plan on working out is just as fair. Have a happy holiday!


#15

Gotta say stuff has improved since Liz got back tbh


#16

Hey @Liz, thank you for taking the time to explain this to us!
I would love to know (and I’m sure many others would, too) at which hours the ranks change. Thanks in advance!


#17

I appreciate you all resolving this quickly. I was disheartened because the glitch happened at a crucial time for me and I lost some momentum. Hopefully the fact that it was only a few days won’t matter too much in the long run - for me and others in the same boat. I appreciate you all listening to us and restoring the system for the time being.


#18

Story which has 10.000 reads by all logic needs 10.000 times more reads to weight same as 100 reads of new published story.

No wonder top stories forever stay top.


#19

They solved this issue within a couple of days but can’t answer our questions or give us information or give us updates when we were asking for representation and diversity from the Diversity thread and them stealing stories? :face_with_monocle: And we have been asking for months yet they do this in less than a week?


#21

4 hours works well for all of those smaller stories and I’m grateful for the explanations!


#22

Thank you for the clarification @Liz. I feel like maybe if we had gotten this explanation when the new ranking system was announced we would have been more accepting towards it. Greatly appreciate all the hard work you guys have put in though