Skip to content

How to design Twitch

The first sections are free. The deep dives, level expectations and quiz are part of Pro.

On this page
Asked at
Practice this question

The problem

A live streaming site takes one feed from a streamer, turns it into short video segments at several quality levels, and hands those segments to a crowd that watches a few seconds behind while chatting with each other. The answer turns on treating the two halves differently: video is a file-delivery problem, where segments and a rolling playlist are cached at a CDN edge and the origin sees each file about once per edge, while chat is a push problem, where one message has to reach a crowd of over 100,000 people within a fraction of a second. The biggest stream decides the design on both sides: it sets how much video leaves the edges and how many servers its chat must spread across.

Clarifying questions

AskWhy it mattersA typical answer

How far behind the streamer may viewers be?

The delay budget sets the segment length, the player buffer and whether plain segmented delivery is enough or partial segments are needed. A broadcast-style delay of half a minute and a near-live delay of a few seconds lead to different designs.

A few seconds. Not sub-second: this is a show with chat, not a call.

How many streams are live at once, and how big is the biggest audience?

Millions of small streams and one giant stream stress different parts: the transcoder fleet grows with the stream count, while the CDN and chat fan-out grow with the biggest audience.

Millions of live streams at once; the biggest draws over 100,000 concurrent viewers.

How fast does chat move on the biggest stream, and must each viewer see each message?

Whether each message must reach everyone decides if sampling or a slow mode is an acceptable lever when the crowd and the message rate are both at their peak.

Thousands of messages a second, up to 5,000 on the biggest streams, and messages should land within 200 ms. On the largest crowds, showing a sample is acceptable if you say so.

Do viewers watch on phones and weak networks, so we need several quality levels?

Several renditions multiply the transcoding work per stream and the number of files the CDN holds, so it changes the cost model.

Yes. Any device and network, so the player should pick a quality that fits its bandwidth.

Do we keep the video after the stream ends?

Recording adds a storage estimate and a second write path from the transcoder, and it decides whether the origin is a short rolling buffer or a durable store.

Not required for this round, but estimate what an hour of video would cost in storage if asked.

Is chat moderation in scope?

Moderation tools are a product of their own; the design question is whether the chat path has a place to drop or delay a message before it fans out.

Out of scope. Leave a hook: the chat service sees each message before it is published.

The requirements menu

RequirementScopeWhy

A streamer broadcasts live audio or video

Core

The input side: an ingest point near the streamer takes one feed and hands it to the transcoders.

Viewers watch the stream on any device and network

Core

Forces several qualities and a player that switches between them, and a delivery path that works through ordinary HTTP.

Viewers post chat messages and see others’ in near real time

Core

The second system inside the product: a write path for posts and a push path that fans each message out to the crowd.

Viewers find live streams to watch

Core

A directory of what is live right now, kept fresh as streams start and end. It gets far less of the round than video or chat, but the product has no front door without it.

Several quality levels that the player switches between as bandwidth changes

Optional

Worth taking: it is how the design serves viewers on weaker networks, and it shapes the transcoder. Leave it if the interviewer says a single quality is fine.

Record the stream so it can be watched after it ends

Optional

The segments already exist, so recording is a second write from the transcoder to durable storage plus a playlist that stops rolling. Take it if asked; it brings a storage estimate with it.

A viewer count on each stream

Optional

Cheap to approximate from the realtime servers’ connection counts and tempting to add, but it is a counting problem with its own fan-in. Mention it and move on unless asked.

Per-viewer message rate limits and a slow mode on the biggest streams

Optional

Take it: it is a design lever as much as a feature. Fewer messages per second on the largest crowds is the cheapest way to keep chat deliverable.

Show the last few messages to a viewer who joins late

Optional

Falls out of the replay buffer the chat path needs anyway for reconnects, so it costs almost nothing once that exists.

Recommendations

Out of scope

A ranking system on top of the directory. Name it and leave it.

Payments, subscriptions and tips

Out of scope

Money paths need their own consistency story, which has nothing to do with video or chat delivery.

Moderation tools

Out of scope

Its own product. Keep the chat service as the single place a message passes before it is published, so moderation has somewhere to plug in later.

Clips and editing

Out of scope

Clipping reads segments that already exist, so it adds a reader of the origin rather than a change to the live path. Name it and move on.

Non-functional requirements

RequirementWhat it forces

Viewers see the stream a few seconds behind the streamer

Short segments, a small player buffer and a transcoder that does not wait for the full broadcast. Each step from camera to player has a slice of the budget, and a segment that arrives late is skipped rather than waited for.

Chat messages reach viewers in under 200 ms (Hello Interview, for live comments)

No polling: messages are pushed over a connection the viewer already holds, and the path from the post to the push is one publish and one hop to the server that holds the viewer. In this design the push does not wait for the write to be confirmed by replicas; the publish happens as soon as the message has an id.

Millions of live streams at once, some with huge audiences

Transcoding is the cost that grows with the stream count, so ingest and transcoding are a fleet that any stream can land on. The huge audiences force a CDN for video and a chat fan-out that spreads one stream over several servers.

Availability over consistency for chat

A message may be delivered to one viewer slightly before another, and a viewer who reconnects may see a message twice or miss one on the edges of the gap. Chat servers keep serving when the message store is slow, and no step blocks on a quorum.

Estimates

QuantityWorkingResultWhat it sizes

Player buffer from the segment length

Players that follow the HLS spec start playback 3 segments behind the newest one. With 2-second segments: 3 × 2 = 6 s. With 6-second segments, the length Apple recommends: 3 × 6 = 18 s. Encoding and transfer add to both.

About 6 s of buffer with 2 s segments, 18 s with 6 s segments, before encode and network time

The segment length: a few seconds of delay is reachable with segments of about 2 s, or with partial segments, and not with 6 s segments.

Requests at the origin for the biggest stream without a CDN

Each player fetches 1 manifest and 1 segment per 2-second segment, about 1 request a second per viewer. 100,000 viewers × 1 = 100,000 requests a second for one stream. Behind a CDN with, say, 100 edge locations, each edge fetches each file about once: 100 edges × 1 request a second = 100 requests a second at the origin.

100,000 requests a second straight at the origin; about 100 a second behind a CDN of 100 edges

Why the manifest and the segments go through a CDN, and why the origin can be small.

Egress for the biggest stream

Assume a top rendition of 6 Mbps for a 1080p60 source. 100,000 viewers × 6 Mbps = 600,000 Mbps = 600 Gbps if all watch the top quality.

About 600 Gbps for one stream at top quality

Why video leaves from CDN edges near viewers, not from your own servers.

Chat deliveries a second on the biggest stream

5,000 messages a second × 100,000 viewers = 500,000,000 deliveries a second.

500 million pushes a second for one stream

Why the biggest streams get sampled chat or a slow mode: the fan-out, not the writes, is the cost.

Chat servers for the biggest stream

100,000 viewers ÷ 100,000 connections a server = 1 server at full load. Run servers at half load so a dead one has somewhere to fail over: 2 servers. If the audience grows a hundredfold to 10,000,000 viewers, 10,000,000 ÷ 100,000 = 100 servers at full load, about 200 at half.

2 servers for today's biggest stream, around 200 at a hundred times the audience

How many realtime servers one stream's routing must spread across, and why one server per stream is not a plan.

Bytes one chat server pushes at full chat to a full house

Assume a message with its metadata is about 100 bytes on the wire. 5,000 messages a second × 100,000 connections × 100 bytes = 50,000,000,000 bytes a second, 50 GB/s on one server.

About 50 GB/s per server

Why no server pushes the full chat of the biggest stream to 100,000 connections: sample the messages or cap the rate on those streams.

Storage per stream-hour at source quality

6 Mbps × 3,600 s = 21,600 Mb = 2,700 MB, about 2.7 GB an hour for the source rendition. Add the other renditions at their own bitrates.

About 2.7 GB per hour of top-quality video

The storage figure the interviewer leaves to you: recording is optional, and this is what each hour costs if you keep it.

Replay buffer per stream

Keep 60 s of chat: 5,000 messages a second × 60 s × 100 bytes = 30,000,000 bytes, about 30 MB for the biggest stream. A stream with a lower message rate needs proportionally less.

About 30 MB per mega-stream for a minute of replay

The replay cache a reconnecting viewer catches up from.

Core entities

EntityWhat it holds

Stream

One live broadcast: who is streaming, whether it is live, and where the feed comes in.

Segment

A few seconds of video in one quality, named by its position in the stream.

Chat message

One message on one stream, with an id that orders it and lets a reconnect resume.

User

A streamer or a viewer. Accounts are ordinary; the design reads the id and a display name.

The API

EndpointNotes

POST /streams → { stream_id, ingest_url, stream_key }

Creates the stream record and picks an ingest point near the caller. The stream key is a secret the streamer’s encoder presents when it pushes; the ingest server checks it before accepting the feed. The record is not live until the feed arrives.

GET /streams/{id} → manifest URL for playback

Returns the CDN URL of the rolling playlist. The player fetches that playlist again each segment, and the segments it names; this endpoint is called once per viewer, so it stays cheap.

POST /streams/{id}/messages { text } → { message_id }

A plain HTTP post, because viewers write rarely next to how much they read. The chat service assigns the id, stores the message, publishes it on the stream’s channel and returns. Rate-limited per user, more tightly on the biggest streams.

GET /streams/{id}/messages/live → SSE stream of new messages

One long-lived response the server writes events to. Each event carries the message id as its event id; when the connection drops, the browser reopens it on its own, and in this design the client passes the last id it saw as a query parameter so the server can resume from the replay buffer.

GET /streams?live=true → streams to watch

Reads the directory of streams whose status is live, sorted by viewer count or recency. In this design it is served from a cached list that is rebuilt periodically rather than read live; nothing here needs to be exact.

The high-level design

RTMPsegmentsStreamerIngest serverTranscoderSegment originViewer player

Step 1 of 4

Ingest, transcode, serve segments

Start with one feed and one viewer. The streamer’s encoder pushes the feed over RTMP to an ingest server chosen near them, which hands it to a transcoder. A transcoder turns the one incoming feed into several renditions, each cut into short segments, and writes them with a rolling playlist that names the newest few. The player reads the playlist, fetches the segments it lists, and comes back for the playlist again each segment. Segments are plain files with fixed names, which is what makes the next step possible. The origin here is a store the transcoder writes to and players read from.

RTMPsegmentsPOSTSSEpublishSSEcatch upStreamerIngest serverTranscoderSegment originCDNViewer playerViewerChat serviceLB, hash ofstreamRealtime serversMessage storePub/sub
The whole design

What breaks first

The deep dives

From the streamer to the viewer. The streamer waves at the camera. Walk one frame through each box until a viewer sees it, say how long each step is allowed to take, and what viewers see when the transcoder for that stream dies.

A concept that helps here: Transcoding and adaptive streamingA concept that helps here: CDNs and edge cachingA concept that helps here: Backpressure and load shedding

Chat at scale. A stream has over 100,000 viewers and thousands of messages a second. Walk one message from the viewer who typed it to the rest of the crowd, say how a chat server knows which messages its viewers need, and what happens to a server’s viewers when it dies.

A concept that helps here: Routing messages between stateful serversA concept that helps here: WebSockets, server-sent events and pollingA concept that helps here: Hot keys and hot partitions

Concepts this guide uses

Sources

  1. Twitch Developers: Video Broadcast
  2. Twitch Developers: Video Broadcast Reference
  3. AWS Media Blog: Demystifying Apple low-latency HTTP live streaming
  4. IETF: HTTP Live Streaming (the HLS RFC)
  5. Redis documentation: Pub/sub
  6. Redis documentation: XREAD
  7. MDN: Using server-sent events
Sign in to report a mistake