How to design Pastebin
The first sections are free. The deep dives, level expectations and quiz are part of Pro.
On this page
- The problem
- Clarifying questions
- The requirements menu
- Non-functional requirements
- Estimates
- Core entities
- The API
- The high-level design
- What breaks first
- The deep dives
- Entity fields
- Generating unique keys
- Storing bodies and serving reads
- Expiry and removal
- Variants
- What each level is expected to show
- The 30-second answer
- Quiz
- BojaleLabs
The problem
A paste service takes a block of text, stores it, and hands back a link; whoever opens the link reads the text until it expires or its owner removes it. Nearly all the traffic is reads, and one paste shared on a busy forum can pull thousands of readers a second onto a single key. The answer turns on three things: giving each paste a key no other paste holds without slowing the write path, serving a viral paste without sending its readers to the stores, and making a removed or expired paste disappear from each copy you made of it.
Clarifying questions
| Ask | Why it matters | A typical answer |
|---|---|---|
How many pastes are created a day, and how many times is a paste read? | The ratio decides whether reads get their own service and a cache, and the daily count sizes the stores and the key space. | The interviewer leaves traffic to you, saying just that reads far outnumber writes. A workable assumption: about 1 million new pastes a day, so about 10 a second, and about 100 reads per paste, so about 1,000 reads a second on average, with a viral paste drawing thousands a second on its own. |
How large can a paste be, and is there a cap? | The body size decides where the text lives: a few hundred bytes could sit in the metadata row, hundreds of kilobytes belong in blob storage. | There is a size limit; pick one and defend it. Pastebin itself caps free pastes at 512 KB, which is a fair reference point. |
How short do the links need to be? | A long random key needs no coordination; a short key needs something that hands out keys without repeats. The length drives the key design. | Keys are 128 bits for this round, which makes a link about 22 characters in |
Can a paste expire, be removed, or be edited? | Expiry and removal decide how long any cache may hold a copy; editing would add a versioning problem. | Expiry is optional per paste, removal is required, and pastes are not edited. |
Who can read a paste, and who can remove one? | Access control changes the read path completely, and removal needs some notion of who is allowed to call delete. | Anyone with the link can read. Sign-in and private sharing are out of scope, so agree on something small for delete, such as a secret returned at create time and kept out of the shared link. |
How quickly must a removed paste stop being readable? | Seconds versus hours decides whether you can rely on cache lifetimes or need an active purge of each copy. | Quickly: the scenario the interviewer has in mind is a password pasted by mistake and deleted a minute later while people keep opening the link. |
Non-functional requirements
| Requirement | What it forces |
|---|---|
Every link is unique: random keys of about 128 bits make a clash negligible | The write path needs a key source that does not repeat. With 128 random bits there is nothing to coordinate; with shorter keys you either check the key on insert and retry a clash, or run a service that hands out ranges. Either way the key column is the primary key, so a repeat is refused rather than overwriting someone's paste. |
A paste body has a size limit | A check on create that rejects oversized text, and a known upper bound for sizing the body store, the cache entries and the bandwidth of a viral paste. |
Reads far outnumber writes | Split the read service from the write service so the read side scales on its own, and put a cache, then a CDN, on the read path. The write path can stay simple. |
A removed or expired paste stops being served quickly | Each copy of a paste must have an owner that can kill it: the row is deleted first, the read service checks the expiry on each read, cache lifetimes are capped at the paste's expiry, and the edge copy is purged on removal. |
Estimates
| Quantity | Working | Result | What it sizes |
|---|---|---|---|
New pastes a second | Assume 1 million new pastes a day; 1 million ÷ 100,000 seconds ≈ 10 a second | About 10 new pastes a second | The write path and the key service: tiny load, so one modest write service and a key service called once per block are plenty of design. |
Reads a second, average and viral | Assume about 100 reads per paste: 100 million reads a day ÷ 100,000 seconds ≈ 1,000 a second on average. For a viral paste take 5,000 reads a second on one key, which is the interviewer's "thousands"; at an average body of 10 KB that is 5,000 × 10 KB = 50 MB a second, about 400 Mbit a second, for that one paste | About 1,000 reads a second on average; a single viral paste can draw 5,000 a second and about 400 Mbit a second of body traffic | The read path: the hot key problem in the cache, and the bandwidth argument for serving bodies from a CDN rather than from your read servers. |
Storage for a year of pastes | Pastebin caps free pastes at 512 KB and Pro pastes at 10 MB; most pastes are far smaller, so assume an average body of 10 KB. 1 million × 10 KB = 10 GB of bodies a day; × 365 days ≈ 3.65 TB a year. Metadata at about 100 bytes a row: 1 million × 100 bytes = 100 MB a day, about 36.5 GB a year | About 3.65 TB of bodies and about 36.5 GB of metadata a year | Why bodies go to blob storage and metadata to a document store: the bodies are a hundred times the metadata, and the metadata for years fits a small replica set. |
Key length in characters | 128 bits is 32 hex digits as a UUID, or 22 | 22 characters for a 128-bit random key, 11 for 64 bits, 8 for a counter with centuries of room | The link length, and the argument for a key service once links must be short. |
Chance of a clash with random keys | Birthday bound: about n² ÷ 2^(b+1) for n keys of b bits. For a billion stored pastes (about 3 years at 1 million a day): 128 bits gives 10^18 ÷ 6.8 × 10^38 ≈ 1.5 × 10^-21; 64 bits gives 10^18 ÷ 3.7 × 10^19 ≈ 2.7% | Negligible at 128 bits; a 2.7% chance of one clash somewhere over a billion pastes at 64 bits | Why 128 random bits need no coordination and no check, and why 64 bits is a judgment call rather than a free shortening. |
Key service calls and wasted keys | Blocks of 1,000 keys: at 10 pastes a second the whole fleet uses 10 × 60 = 600 keys a minute, so about 0.6 block requests a minute in total, however many write servers share the load (one server would ask once per 100 seconds). A server that fetches its next block when the current one is half used holds up to about 500 + 1,000 = 1,500 unissued keys, so a crash wastes at most about 1,500 keys of about 218 trillion | Under one block request a minute in total; a crash wastes at most about 1,500 keys, a negligible fraction of the key space | Why the key service is a light service, and why losing a block on a crash is acceptable. |
How long a daily sweeper over-serves a short paste | A paste set to expire in an hour, 3,600 seconds, waits for a sweeper that runs once per about 100,000 seconds: up to 100,000 ÷ 3,600 ≈ 28 times its intended life | Up to about 28 hours of serving a paste meant to last 1 hour | Why expiry has to be checked on the read path and not left to a background job. |
Cache memory for the hot set | The 10,000 hottest pastes at 10 KB each = 100 MB; even a whole day of bodies, 10 GB, is a cache-node-sized amount | About 100 MB for the hot set | Why the size of the cache is not the concern; the single hottest key and its bandwidth are. |
Core entities
| Entity | What it holds |
|---|---|
Paste | One document per paste: the key, where its text lives, and when it stops being readable. |
Key block | The key service's whole state: where the next unissued range of keys begins. |
The API
| Endpoint | Notes |
|---|---|
| Reject text over the cap with |
| The hot path. Check the cache, then the metadata row, then fetch the body; compare |
| Delete the metadata row first, evict the cache key, request a purge of the edge URL, then queue the body delete for the sweeper to carry out and retry. The order shrinks what a reader can see at each step. A reader already mid-request can still refill the cache from the row it read before the delete, so instead of a plain eviction write a short-lived 'gone' marker under the key, which a late refill does not overwrite. Idempotent: a second delete of the same key returns the same success. |
The high-level design
Step 1 of 5
Two services, two stores
Start with the two jobs. A write service takes POST /pastes, checks the size, stores the text and inserts a metadata row. A read service takes GET /pastes/{key}, reads the row, then the text. They are separate because reads far outnumber writes: the read side scales on its own and a bad deploy on the write side does not take reads down. The text goes to blob storage as an object, and the metadata (key, body location, expiry) goes to a document store, which fits because pastes need no joins or transactions. At the assumed traffic that is about 3.65 TB of bodies a year against about 36.5 GB of metadata.
What breaks first
The deep dives
Generating unique keys. Two people create pastes in the same instant and each must get a key nobody else holds. How does a new paste get its key, how long does the key have to be, and what happens to new pastes when the part that makes keys is down?
Storing bodies and serving reads. A paste lands on a busy forum and draws thousands of reads a second. Walk one read from the browser to the text, say which box fails first, and explain what you put in front of it and where the body lives.
Expiry and removal. Someone pastes a password by mistake and deletes the paste a minute later while people keep opening the link. How does it stop being served from each copy, and how do expired pastes get found and cleaned up when a body delete fails?
Concepts this guide uses
Sources
- interviewing.io: A Senior Engineer's Guide to the System Design Interview, part four (Pastebin and the Key Generation Service)
- Wikipedia: Universally unique identifier
- Pastebin.com: FAQ
- Amazon S3 User Guide: Downloading objects
- MDN: Cache-Control header
- Redis documentation: EXPIRE
- Redis cluster specification
- Amazon ElastiCache documentation: Caching strategies
- MongoDB manual: TTL Indexes
- MongoDB manual: Replication and Replica Set Fundamentals
- Cloudflare Cache docs: Purge by single-file
- Educative: Pastebin System Design
- System Design School: Design Pastebin