How a "hot" ranking works

Ken Shirriff has a great post this month on how Hacker News ranking really works. He scraped the front page every minute and compared where stories actually ranked against where the published formula said they should. The answer is that there’s a simple formula, plus a surprising number of penalties layered on top.

We needed a “hot” list for a feed in our own product recently, so I’d been reading up on exactly this. Here’s what I learned, and what we ended up doing.

What a hot list is for

You want the front page to show things that are both good and new. Sort by votes and the front page never changes, because old stories have had longer to collect votes. Sort by date and you show whatever was posted last, good or not. A hot ranking is a way of trading votes against age.

Hacker News: gravity

The core formula, from the Arc source code that HN runs on:

score = (votes - 1) / (age_in_hours + 2) ^ gravity

with gravity at 1.8. The - 1 removes the submitter’s own vote. The + 2 stops brand-new stories from dividing by nearly zero and shooting to the top. And gravity controls how quickly age wins. Here’s a story with 100 votes, scored over a day, at two different gravities:

At gravity 1.8, that 100-vote story has lost over 90% of its score by hour eight. That’s why the HN front page turns over so quickly. A quick example of what that means in practice:

• Story A: 200 votes, posted 10 hours ago. 199 / 12^1.8 = 2.27
• Story B: 30 votes, posted 1 hour ago. 29 / 3^1.8 = 4.01

B ranks above A, despite having a seventh of the votes. A gravity that high says “we’d rather show you something new and pretty good than something old and great”.

Shirriff’s post also shows that the formula is only part of the story. Stories get penalties for things like having many more comments than votes, which is how HN detects flamewars, and for certain domains. Some stories, it seems, are penalized by hand. If you’ve ever wondered why a highly voted story suddenly fell off the front page, that’s probably why.

Reddit: logarithms and a fixed epoch

Reddit’s “hot” ranking, which is in its open-source code, takes a different approach:

from math import log10

def hot(ups, downs, posted_seconds):
    s = ups - downs
    order = log10(max(abs(s), 1))
    sign = 1 if s > 0 else -1 if s < 0 else 0
    seconds = posted_seconds - 1134028003   # a fixed date in December 2005
    return round(sign * order + seconds / 45000, 7)

Two things are different here. First, votes count logarithmically: going from 10 to 100 net votes is worth exactly as much as going from 100 to 1,000. The first votes matter far more than the later ones. Second, there’s no decay at all. A story’s score never changes with age. Instead, newer stories get a bigger bonus, because seconds keeps growing. Every 45,000 seconds, about 12.5 hours, adds one point, which is the same as a 10x increase in votes. The effect is the same as decay, but the scores are stable, so you can store them and index them in the database instead of recomputing them on every page view.

What we used

Our feed is small: hundreds of items a day, not hundreds of thousands. We ended up with something closer to Reddit’s design, for one practical reason. Because the score only changes when someone votes, we can compute it on write, store it in a column, and serve the hot list from a plain ORDER BY on an index. With the HN formula, every item’s score changes every minute, so you have to recompute it constantly or do it at query time.

We did tune it:

• A much bigger divisor than 45,000. Our items stay relevant for days, not hours, so it takes a lot longer for new to beat good.
• Logarithmic votes. This keeps a single very popular item from sitting at the top forever.
• A small boost for brand-new items so they get seen at all. That’s the opposite of HN’s + 2, which holds new stories back. Our feed is small enough that we want new things to get a look.

One more thing worth knowing about: if you’re ranking by rating rather than freshness (“best”, not “hot”), don’t sort by average. One five-star review beats a hundred 4.9s. Evan Miller’s How Not To Sort By Average Rating explains using the lower bound of the Wilson score interval instead, which is what Reddit uses for its “best” comment sort. Read it before you build a ratings page.