Track 2 · Products people use
mediumNews Feed
Show each user the recent posts of the people they follow. The whole design is one question: do the work when a post is written, or when a feed is read?
Suggested architecture
Scenario
Fan-out on write makes reads cheap by making every post expensive: one post becomes a write per follower. Turn followers per post up and watch the cache and the workers absorb it — until a celebrity posts.
Feeds opened per second. Each is one cache read plus hydration.
New posts per second, each fanned out to its author's followers.
Average follower count of whoever is posting. The multiplier on every write.
Feeds served from a precomputed list. A miss rebuilds the timeline from the store — twenty queries, not one.
Click a component for its role, common technology choices and tradeoffs, and what it is carrying at this scale. Hover a connection to see what flows along it. Drag to rearrange — layout changes are local and reset on reload.
Every figure here is a rough estimate from simple capacity arithmetic, not a benchmark. Each part carries its own assumption about what one copy can do — real numbers depend on your hardware, payloads and access pattern. The point is which component moves first as you turn the dials, not the digits themselves.
In plain words
A news feed shows you the latest posts from the people you follow. Computing that on every refresh means combining hundreds of lists, and people refresh far more often than they post — so the design does the combining once, when a post is made, instead of every time someone looks. Then it deals with the one case where that is impossible: an account with twenty million followers.
The shape of the problem
A feed is a join (combine several lists into one): for the caller, find everyone they follow, take those people's recent posts, sort by time, return the top fifty. Done naively at read time that is one query per followed account, and a user who follows 500 people costs 500 queries per refresh. With reads outnumbering posts by a hundred to one, the read path is where all the money goes — so the design moves the work to the write path.
Assume 10 M daily users opening the feed 30 times a day, and posting 3 times.
- Reads: 300 M / 86 400 ≈ 3 500 feed/s average; take 30 000 at peak.
- Posts: 30 M / 86 400 ≈ 350 posts/s.
- Followers: a median user has 200; a few have 20 million.
The asymmetry is the design. Three hundred writes a second can afford to be expensive. Thirty thousand reads a second cannot.
Fan-out on write
When a user posts, a worker reads their follower list and appends the post id to each follower's timeline — a per-user list in Redis, capped at a few hundred entries. A feed read is then one list fetch plus a batched hydration — fetching the full post for each id — of the post bodies. No join, no sorting, no query across followed accounts.
The price is paid at write time: a post by someone with 200 followers is 200 list appends. At 350 posts/s that is 70 000 appends/s, which a small Redis fleet handles without noticing. Turn the followers per post slider up and watch where it stops being small.
The timeline is a cache, not a store. It can be evicted, rebuilt, or wrong for a few seconds, and nobody will notice. The post store is the truth; the timeline is how you avoid asking it.
The celebrity problem
Fan-out on write breaks the moment one account has enough followers. A post from an account with 20 M followers is 20 M appends — minutes of work for the worker pool, during which every other post queues behind it, and a burst of celebrity posts is an outage.
Every real feed uses a hybrid. Accounts over a follower threshold (say 100 000) are not fanned out. Their posts go only to the post store, and the feed service merges them in at read time: fetch the caller's precomputed timeline, fetch recent posts from the handful of celebrities they follow, merge, return. Reads pay a small fixed cost for a large variable one that writes could not pay at all.
Keeping media off the path
Images and video are most of the bytes and none of the logic. They go straight from the client to object storage on upload and straight from the CDN (a content delivery network: caches placed close to users) to the client on view; the feed carries URLs. A feed page with twenty images is one API call and twenty CDN hits, and the origin behind the CDN — your own storage — sees only the misses.
Where this design breaks
- Timeline cache misses. An evicted (dropped to make room) timeline is rebuilt with a query per followed account — the naive read path, twenty times more expensive. Keep the hit rate high or the store takes the full read load on a cold restart.
- Ranking. The design above orders by time. A ranked feed scores candidates at read time, which brings compute back to the hot path and is why ranked feeds precompute scores in the fan-out step.
- Follower list reads. The worker reads the poster's follower list for every post. Cache it; it changes slowly and is read constantly.
- Deletes and edits. A deleted post is already in a million timelines. Filter at read time rather than trying to un-fan-out.
Take this with you
- The one idea: move work from the busy path (reads) to the quiet path (writes) — and switch back to read-time merging for the few accounts where the write-time cost explodes.
- In an interview, say "fan-out on write, hybrid for celebrities", give the numbers, and point out that the timeline is a cache, not the truth.
- At work, measure your timeline hit rate. A cold cache turns every read into the expensive path you designed away.