Track 3 · Scale machinery
hardVideo Streaming
Upload once, transcode into a ladder of qualities, and serve millions of viewers from the edge. Bytes are the problem; the API is an afterthought.
Suggested architecture
Scenario
Viewers pull bytes from the edge; your servers see manifests and misses. The two numbers that decide the bill are the CDN hit rate and how many transcoders it takes to keep up with uploads that each cost minutes of CPU.
Sessions starting per second. Each pulls a manifest, then segments — the fanout on the CDN edge.
Share of segment requests the edge answers. Popularity is extreme, so this is high — and every point below it is origin bandwidth.
Videos uploaded per second. Each is minutes of transcoding.
Each holds two jobs of about two minutes. Uploads per second × 120 s ÷ 2 is how many you need — hundreds, not tens.
Click a component for its role, common technology choices and tradeoffs, and what it is carrying at this scale. Hover a connection to see what flows along it. Drag to rearrange — layout changes are local and reset on reload.
Every figure here is a rough estimate from simple capacity arithmetic, not a benchmark. Each part carries its own assumption about what one copy can do — real numbers depend on your hardware, payloads and access pattern. The point is which component moves first as you turn the dials, not the digits themselves.
In plain words
A video service stores uploads in several qualities and plays them back to millions of viewers, each of whom picks the quality their connection can handle. The surprise is where the work is: the API is tiny, the bytes are enormous, and the design is really about making sure those bytes come from a cache near the viewer instead of from you.
The shape of the problem
Video is the one system where the API is genuinely the small part. A watch is a manifest — a table of contents, a few kilobytes of JSON, listing every segment and quality — followed by hundreds of segment fetches, each a megabyte or two, for as long as the viewer keeps watching. Multiply by millions of viewers and the bytes dwarf everything else by four orders of magnitude (ten thousand times). The design is about where those bytes come from, and it is almost never your servers.
Assume 20 000 sessions starting per second at peak, each pulling a 4-second segment at a time — fifteen a minute — at an average 3 Mbps, and 5 uploads a second of videos that each take minutes to transcode.
- Segment fetches: 20 000 × 15 / 60 ≈ 5 000 segments/s … per second of session starts, compounding for as long as sessions last.
- Egress (bytes leaving your network — what the cloud bills you for): a few hundred gigabits per second from the edge.
- Transcoding: 5 uploads/s × 2 minutes each = 600 videos in flight at any moment.
The CDN is the product
Viewing is a cache problem with a very skewed key distribution. A few hundred videos are most of the views on any given day, so an edge cache gets a hit rate in the high nineties without trying. Segments are immutable — a transcoded chunk never changes once written — so there is no invalidation, and cache lifetimes are effectively forever.
That leaves the origin — object storage holding every quality of every video — serving only misses: the first viewer of anything, and the long tail. Move the hit rate slider and watch what a single percentage point does to the segment store. Each point is a percent of a few hundred gigabits, which is why origin egress is the line item that decides whether the business works.
Serve bytes from the edge, decisions from the API. The playback API hands out a manifest and records progress; it never sees a video byte.
Adaptive bitrate is the player's job
Each video is transcoded into a ladder — 240p up to 4K — and every quality is cut into segments at the same time boundaries. The manifest lists all of them. The player measures how fast the last segment arrived and picks the quality of the next one accordingly, stepping down before it stalls and up when there is room.
The server's role in this is to have every quality ready. Adaptation happens entirely on the client, per segment, which is why the design has no "quality selection service": there is nothing to decide server-side.
Transcoding is the slow, expensive, boring part
An upload is the one write path, and it is tiny in count and enormous in work. A ten-minute video into six qualities is minutes of CPU per copy. The transcoder is the only part in the library where an instance handles less than one request a second: a copy that holds two jobs of two minutes each finishes one every sixty seconds, and the model sizes it by slots held rather than by throughput.
Two things make it tolerable. Uploads go straight from the creator to object storage via a signed URL (a link that lets one client write one object, for a short time), so no server proxies gigabytes. And the transcode job goes on a queue, so the upload finishes in seconds and the minutes happen elsewhere. To make the minutes shorter, split the source into chunks and transcode them in parallel; the ladder for a long video can be ready in under a minute.
Where this design breaks
- A new video going viral. The first thousand viewers all miss the edge at once, and object storage allows only a few thousand requests a second per prefix (a folder-like key range). Spread segment keys across prefixes and let the CDN collapse identical simultaneous requests into one.
- Live streaming. Segments are produced as the event happens; there is no "ready", and the edge must fetch each one within seconds of it existing. A different design with the same parts.
- Progress writes. Every viewer reporting position every few seconds is a write rate larger than any other in the system. It goes to a pub/sub topic, not the database, and is folded down before it lands.
- Storage. Six qualities of everything ever uploaded, forever. Tiered storage and deleting the 4K copy of videos nobody watches is a real policy, not an optimisation.
Take this with you
- The one idea: serve bytes from the edge, decisions from the API. Cache hit rate is the number that decides whether the business works.
- In an interview, describe the segment + manifest model, adaptive bitrate on the client, and the upload → queue → transcode path.
- At work, anything with a heavily skewed, immutable read set (thumbnails, package downloads, map tiles) is this design. Put a CDN in front and watch the origin traffic vanish.