How to design Twitch
The first sections are free. The deep dives, level expectations and quiz are part of Pro.
On this page
The problem
A live streaming site takes one feed from a streamer, turns it into short video segments at several quality levels, and hands those segments to a crowd that watches a few seconds behind while chatting with each other. The answer turns on treating the two halves differently: video is a file-delivery problem, where segments and a rolling playlist are cached at a CDN edge and the origin sees each file about once per edge, while chat is a push problem, where one message has to reach a crowd of over 100,000 people within a fraction of a second. The biggest stream decides the design on both sides: it sets how much video leaves the edges and how many servers its chat must spread across.
Clarifying questions
| Ask | Why it matters | A typical answer |
|---|---|---|
How far behind the streamer may viewers be? | The delay budget sets the segment length, the player buffer and whether plain segmented delivery is enough or partial segments are needed. A broadcast-style delay of half a minute and a near-live delay of a few seconds lead to different designs. | A few seconds. Not sub-second: this is a show with chat, not a call. |
How many streams are live at once, and how big is the biggest audience? | Millions of small streams and one giant stream stress different parts: the transcoder fleet grows with the stream count, while the CDN and chat fan-out grow with the biggest audience. | Millions of live streams at once; the biggest draws over 100,000 concurrent viewers. |
How fast does chat move on the biggest stream, and must each viewer see each message? | Whether each message must reach everyone decides if sampling or a slow mode is an acceptable lever when the crowd and the message rate are both at their peak. | Thousands of messages a second, up to 5,000 on the biggest streams, and messages should land within 200 ms. On the largest crowds, showing a sample is acceptable if you say so. |
Do viewers watch on phones and weak networks, so we need several quality levels? | Several renditions multiply the transcoding work per stream and the number of files the CDN holds, so it changes the cost model. | Yes. Any device and network, so the player should pick a quality that fits its bandwidth. |
Do we keep the video after the stream ends? | Recording adds a storage estimate and a second write path from the transcoder, and it decides whether the origin is a short rolling buffer or a durable store. | Not required for this round, but estimate what an hour of video would cost in storage if asked. |
Is chat moderation in scope? | Moderation tools are a product of their own; the design question is whether the chat path has a place to drop or delay a message before it fans out. | Out of scope. Leave a hook: the chat service sees each message before it is published. |
Non-functional requirements
| Requirement | What it forces |
|---|---|
Viewers see the stream a few seconds behind the streamer | Short segments, a small player buffer and a transcoder that does not wait for the full broadcast. Each step from camera to player has a slice of the budget, and a segment that arrives late is skipped rather than waited for. |
Chat messages reach viewers in under 200 ms (Hello Interview, for live comments) | No polling: messages are pushed over a connection the viewer already holds, and the path from the post to the push is one publish and one hop to the server that holds the viewer. In this design the push does not wait for the write to be confirmed by replicas; the publish happens as soon as the message has an id. |
Millions of live streams at once, some with huge audiences | Transcoding is the cost that grows with the stream count, so ingest and transcoding are a fleet that any stream can land on. The huge audiences force a CDN for video and a chat fan-out that spreads one stream over several servers. |
Availability over consistency for chat | A message may be delivered to one viewer slightly before another, and a viewer who reconnects may see a message twice or miss one on the edges of the gap. Chat servers keep serving when the message store is slow, and no step blocks on a quorum. |
Estimates
| Quantity | Working | Result | What it sizes |
|---|---|---|---|
Player buffer from the segment length | Players that follow the HLS spec start playback 3 segments behind the newest one. With 2-second segments: 3 × 2 = 6 s. With 6-second segments, the length Apple recommends: 3 × 6 = 18 s. Encoding and transfer add to both. | About 6 s of buffer with 2 s segments, 18 s with 6 s segments, before encode and network time | The segment length: a few seconds of delay is reachable with segments of about 2 s, or with partial segments, and not with 6 s segments. |
Requests at the origin for the biggest stream without a CDN | Each player fetches 1 manifest and 1 segment per 2-second segment, about 1 request a second per viewer. 100,000 viewers × 1 = 100,000 requests a second for one stream. Behind a CDN with, say, 100 edge locations, each edge fetches each file about once: 100 edges × 1 request a second = 100 requests a second at the origin. | 100,000 requests a second straight at the origin; about 100 a second behind a CDN of 100 edges | Why the manifest and the segments go through a CDN, and why the origin can be small. |
Egress for the biggest stream | Assume a top rendition of 6 Mbps for a | About 600 Gbps for one stream at top quality | Why video leaves from CDN edges near viewers, not from your own servers. |
Chat deliveries a second on the biggest stream | 5,000 messages a second × 100,000 viewers = 500,000,000 deliveries a second. | 500 million pushes a second for one stream | Why the biggest streams get sampled chat or a slow mode: the fan-out, not the writes, is the cost. |
Chat servers for the biggest stream | 100,000 viewers ÷ 100,000 connections a server = 1 server at full load. Run servers at half load so a dead one has somewhere to fail over: 2 servers. If the audience grows a hundredfold to 10,000,000 viewers, 10,000,000 ÷ 100,000 = 100 servers at full load, about 200 at half. | 2 servers for today's biggest stream, around 200 at a hundred times the audience | How many realtime servers one stream's routing must spread across, and why one server per stream is not a plan. |
Bytes one chat server pushes at full chat to a full house | Assume a message with its metadata is about 100 bytes on the wire. 5,000 messages a second × 100,000 connections × 100 bytes = 50,000,000,000 bytes a second, 50 GB/s on one server. | About 50 GB/s per server | Why no server pushes the full chat of the biggest stream to 100,000 connections: sample the messages or cap the rate on those streams. |
Storage per stream-hour at source quality | 6 Mbps × 3,600 s = 21,600 Mb = 2,700 MB, about 2.7 GB an hour for the source rendition. Add the other renditions at their own bitrates. | About 2.7 GB per hour of top-quality video | The storage figure the interviewer leaves to you: recording is optional, and this is what each hour costs if you keep it. |
Replay buffer per stream | Keep 60 s of chat: 5,000 messages a second × 60 s × 100 bytes = 30,000,000 bytes, about 30 MB for the biggest stream. A stream with a lower message rate needs proportionally less. | About 30 MB per mega-stream for a minute of replay | The replay cache a reconnecting viewer catches up from. |
Core entities
| Entity | What it holds |
|---|---|
Stream | One live broadcast: who is streaming, whether it is live, and where the feed comes in. |
Segment | A few seconds of video in one quality, named by its position in the stream. |
Chat message | One message on one stream, with an id that orders it and lets a reconnect resume. |
User | A streamer or a viewer. Accounts are ordinary; the design reads the id and a display name. |
The API
| Endpoint | Notes |
|---|---|
| Creates the stream record and picks an ingest point near the caller. The stream key is a secret the streamer’s encoder presents when it pushes; the ingest server checks it before accepting the feed. The record is not live until the feed arrives. |
| Returns the CDN URL of the rolling playlist. The player fetches that playlist again each segment, and the segments it names; this endpoint is called once per viewer, so it stays cheap. |
| A plain HTTP post, because viewers write rarely next to how much they read. The chat service assigns the id, stores the message, publishes it on the stream’s channel and returns. Rate-limited per user, more tightly on the biggest streams. |
| One long-lived response the server writes events to. Each event carries the message id as its event id; when the connection drops, the browser reopens it on its own, and in this design the client passes the last id it saw as a query parameter so the server can resume from the replay buffer. |
| Reads the directory of streams whose status is live, sorted by viewer count or recency. In this design it is served from a cached list that is rebuilt periodically rather than read live; nothing here needs to be exact. |
The high-level design
Step 1 of 4
Ingest, transcode, serve segments
Start with one feed and one viewer. The streamer’s encoder pushes the feed over RTMP to an ingest server chosen near them, which hands it to a transcoder. A transcoder turns the one incoming feed into several renditions, each cut into short segments, and writes them with a rolling playlist that names the newest few. The player reads the playlist, fetches the segments it lists, and comes back for the playlist again each segment. Segments are plain files with fixed names, which is what makes the next step possible. The origin here is a store the transcoder writes to and players read from.
What breaks first
The deep dives
From the streamer to the viewer. The streamer waves at the camera. Walk one frame through each box until a viewer sees it, say how long each step is allowed to take, and what viewers see when the transcoder for that stream dies.
Chat at scale. A stream has over 100,000 viewers and thousands of messages a second. Walk one message from the viewer who typed it to the rest of the crowd, say how a chat server knows which messages its viewers need, and what happens to a server’s viewers when it dies.