Skip to content

How to design Pastebin

The first sections are free. The deep dives, level expectations and quiz are part of Pro.

On this page
Asked at
  • BojaleLabs
Practice this question

The problem

A paste service takes a block of text, stores it, and hands back a link; whoever opens the link reads the text until it expires or its owner removes it. Nearly all the traffic is reads, and one paste shared on a busy forum can pull thousands of readers a second onto a single key. The answer turns on three things: giving each paste a key no other paste holds without slowing the write path, serving a viral paste without sending its readers to the stores, and making a removed or expired paste disappear from each copy you made of it.

Clarifying questions

AskWhy it mattersA typical answer

How many pastes are created a day, and how many times is a paste read?

The ratio decides whether reads get their own service and a cache, and the daily count sizes the stores and the key space.

The interviewer leaves traffic to you, saying just that reads far outnumber writes. A workable assumption: about 1 million new pastes a day, so about 10 a second, and about 100 reads per paste, so about 1,000 reads a second on average, with a viral paste drawing thousands a second on its own.

How large can a paste be, and is there a cap?

The body size decides where the text lives: a few hundred bytes could sit in the metadata row, hundreds of kilobytes belong in blob storage.

There is a size limit; pick one and defend it. Pastebin itself caps free pastes at 512 KB, which is a fair reference point.

How short do the links need to be?

A long random key needs no coordination; a short key needs something that hands out keys without repeats. The length drives the key design.

Keys are 128 bits for this round, which makes a link about 22 characters in base62. Be ready to say what you would change if people wanted links short enough to read aloud.

Can a paste expire, be removed, or be edited?

Expiry and removal decide how long any cache may hold a copy; editing would add a versioning problem.

Expiry is optional per paste, removal is required, and pastes are not edited.

Who can read a paste, and who can remove one?

Access control changes the read path completely, and removal needs some notion of who is allowed to call delete.

Anyone with the link can read. Sign-in and private sharing are out of scope, so agree on something small for delete, such as a secret returned at create time and kept out of the shared link.

How quickly must a removed paste stop being readable?

Seconds versus hours decides whether you can rely on cache lifetimes or need an active purge of each copy.

Quickly: the scenario the interviewer has in mind is a password pasted by mistake and deleted a minute later while people keep opening the link.

The requirements menu

RequirementScopeWhy

Create a paste from a block of text and get back a unique link

Core

The write path: take the text, mint a key, store body and metadata, return the link.

Anyone with the link can read the paste

Core

The read path, which carries nearly all the traffic and is where the design effort goes.

Optionally set an expiry, after which the paste is gone

Core

It puts a deadline on each copy of the paste: the row, the body, the cache entry and the edge copy.

Remove a paste; pastes are never edited

Core

Removal must reach each copy; no edits means a cached body is correct for as long as the paste lives, which keeps caching simple.

Reject a paste over the size cap

Optional

Cheap to take: a check on the write path that returns 413 Content Too Large. It also tells the interviewer you know abuse arrives through the biggest input.

Burn after reading: delete a paste after its first view

Optional

A good probe of your caching story, since a paste that dies on first read must not be cached at all, and the two-readers race has to be settled in the store. Take it if asked; it is a few lines once removal works.

Rate-limit paste creation per client

Optional

Worth a sentence: anonymous creation invites spam, and a limit per IP or token is the usual first defense. Leave the mechanism to a rate-limiting discussion.

Custom, human-chosen keys

Optional

Tempting because readable links are nicer to share. Leave it unless asked: a chosen key and a generated key share one namespace, and that adds a uniqueness check on the write path you otherwise avoid.

Syntax highlighting and rendering of the text

Out of scope

A presentation concern for the client. Store the text and an optional language tag, and let the browser render.

Editing a paste

Out of scope

An edit would invalidate each cached copy and invite version questions; the product says pastes are immutable.

Private sharing, sign-in and access control

Out of scope

It would turn each read into an authorization check and make shared caching much harder, since an edge would have to know who is asking before answering. A natural twist on this question; see the variants.

View analytics and telemetry

Out of scope

A separate pipeline. Note that reads served from a CDN edge are not seen by your servers, so counting views would change the caching story.

GDPR and other regulation

Out of scope

Real for a public text site, but a legal and operational topic. The removal path you design is what a takedown would use.

Non-functional requirements

RequirementWhat it forces

Every link is unique: random keys of about 128 bits make a clash negligible

The write path needs a key source that does not repeat. With 128 random bits there is nothing to coordinate; with shorter keys you either check the key on insert and retry a clash, or run a service that hands out ranges. Either way the key column is the primary key, so a repeat is refused rather than overwriting someone's paste.

A paste body has a size limit

A check on create that rejects oversized text, and a known upper bound for sizing the body store, the cache entries and the bandwidth of a viral paste.

Reads far outnumber writes

Split the read service from the write service so the read side scales on its own, and put a cache, then a CDN, on the read path. The write path can stay simple.

A removed or expired paste stops being served quickly

Each copy of a paste must have an owner that can kill it: the row is deleted first, the read service checks the expiry on each read, cache lifetimes are capped at the paste's expiry, and the edge copy is purged on removal.

Estimates

QuantityWorkingResultWhat it sizes

New pastes a second

Assume 1 million new pastes a day; 1 million ÷ 100,000 seconds ≈ 10 a second

About 10 new pastes a second

The write path and the key service: tiny load, so one modest write service and a key service called once per block are plenty of design.

Reads a second, average and viral

Assume about 100 reads per paste: 100 million reads a day ÷ 100,000 seconds ≈ 1,000 a second on average. For a viral paste take 5,000 reads a second on one key, which is the interviewer's "thousands"; at an average body of 10 KB that is 5,000 × 10 KB = 50 MB a second, about 400 Mbit a second, for that one paste

About 1,000 reads a second on average; a single viral paste can draw 5,000 a second and about 400 Mbit a second of body traffic

The read path: the hot key problem in the cache, and the bandwidth argument for serving bodies from a CDN rather than from your read servers.

Storage for a year of pastes

Pastebin caps free pastes at 512 KB and Pro pastes at 10 MB; most pastes are far smaller, so assume an average body of 10 KB. 1 million × 10 KB = 10 GB of bodies a day; × 365 days ≈ 3.65 TB a year. Metadata at about 100 bytes a row: 1 million × 100 bytes = 100 MB a day, about 36.5 GB a year

About 3.65 TB of bodies and about 36.5 GB of metadata a year

Why bodies go to blob storage and metadata to a document store: the bodies are a hundred times the metadata, and the metadata for years fits a small replica set.

Key length in characters

128 bits is 32 hex digits as a UUID, or 22 base62 characters (62^22 ≈ 2.7 × 10^39 > 2^128 ≈ 3.4 × 10^38). 64 bits needs 11 base62 characters. A counter encoded in 8 base62 characters has 62^8 ≈ 2.18 × 10^14, about 218 trillion keys; at 365 million new pastes a year that is about 600,000 years of keys

22 characters for a 128-bit random key, 11 for 64 bits, 8 for a counter with centuries of room

The link length, and the argument for a key service once links must be short.

Chance of a clash with random keys

Birthday bound: about n² ÷ 2^(b+1) for n keys of b bits. For a billion stored pastes (about 3 years at 1 million a day): 128 bits gives 10^18 ÷ 6.8 × 10^38 ≈ 1.5 × 10^-21; 64 bits gives 10^18 ÷ 3.7 × 10^19 ≈ 2.7%

Negligible at 128 bits; a 2.7% chance of one clash somewhere over a billion pastes at 64 bits

Why 128 random bits need no coordination and no check, and why 64 bits is a judgment call rather than a free shortening.

Key service calls and wasted keys

Blocks of 1,000 keys: at 10 pastes a second the whole fleet uses 10 × 60 = 600 keys a minute, so about 0.6 block requests a minute in total, however many write servers share the load (one server would ask once per 100 seconds). A server that fetches its next block when the current one is half used holds up to about 500 + 1,000 = 1,500 unissued keys, so a crash wastes at most about 1,500 keys of about 218 trillion

Under one block request a minute in total; a crash wastes at most about 1,500 keys, a negligible fraction of the key space

Why the key service is a light service, and why losing a block on a crash is acceptable.

How long a daily sweeper over-serves a short paste

A paste set to expire in an hour, 3,600 seconds, waits for a sweeper that runs once per about 100,000 seconds: up to 100,000 ÷ 3,600 ≈ 28 times its intended life

Up to about 28 hours of serving a paste meant to last 1 hour

Why expiry has to be checked on the read path and not left to a background job.

Cache memory for the hot set

The 10,000 hottest pastes at 10 KB each = 100 MB; even a whole day of bodies, 10 GB, is a cache-node-sized amount

About 100 MB for the hot set

Why the size of the cache is not the concern; the single hottest key and its bandwidth are.

Core entities

EntityWhat it holds

Paste

One document per paste: the key, where its text lives, and when it stops being readable.

Key block

The key service's whole state: where the next unissued range of keys begins.

The API

EndpointNotes

POST /pastes { text, expires_at? } → { url }

Reject text over the cap with 413 Content Too Large before doing anything else. Take the next key from the server's in-memory block, upload the body to blob storage, then insert the metadata row, so no row is written for a body that failed to land. Return the link and, if you adopted it, the delete secret.

GET /pastes/{key} → { text }

The hot path. Check the cache, then the metadata row, then fetch the body; compare expires_at with the clock before answering. Unknown, expired and removed pastes all return HTTP 404 so the response does not confirm that a removed paste existed. The response carries Cache-Control: s-maxage capped at the time left before expiry.

DELETE /pastes/{key}

Delete the metadata row first, evict the cache key, request a purge of the edge URL, then queue the body delete for the sweeper to carry out and retry. The order shrinks what a reader can see at each step. A reader already mid-request can still refill the cache from the row it read before the delete, so instead of a plain eviction write a short-lived 'gone' marker under the key, which a late refill does not overwrite. Idempotent: a second delete of the same key returns the same success.

The high-level design

createrowbodyrowbodyreadClientWrite serviceRead serviceMetadataBlobs

Step 1 of 5

Two services, two stores

Start with the two jobs. A write service takes POST /pastes, checks the size, stores the text and inserts a metadata row. A read service takes GET /pastes/{key}, reads the row, then the text. They are separate because reads far outnumber writes: the read side scales on its own and a bad deploy on the write side does not take reads down. The text goes to blob storage as an object, and the metadata (key, body location, expiry) goes to a document store, which fits because pastes need no joins or transactions. At the assumed traffic that is about 3.65 TB of bodies a year against about 36.5 GB of metadata.

POST /pastesGET /pastes/keymisskey blocksmetadata rowbodyrow, expirybodycheck firstevictpurgeexpired rowsdelete bodiesClientWrite serviceCDN edgeRead serviceKey serviceDocument storeBlob storageRedis cacheExpiry sweeper
The whole design

What breaks first

The deep dives

Generating unique keys. Two people create pastes in the same instant and each must get a key nobody else holds. How does a new paste get its key, how long does the key have to be, and what happens to new pastes when the part that makes keys is down?

A concept that helps here: Unique ID generationA concept that helps here: Leader election and failoverA concept that helps here: Stating a trade-off

Storing bodies and serving reads. A paste lands on a busy forum and draws thousands of reads a second. Walk one read from the browser to the text, say which box fails first, and explain what you put in front of it and where the body lives.

A concept that helps here: Caching strategiesA concept that helps here: CDNs and edge cachingA concept that helps here: Blob storage and presigned uploads

Expiry and removal. Someone pastes a password by mistake and deletes the paste a minute later while people keep opening the link. How does it stop being served from each copy, and how do expired pastes get found and cleaned up when a body delete fails?

A concept that helps here: Cache invalidation and TTLsA concept that helps here: Job scheduling and delayed workA concept that helps here: Retries and dead-letter queues

Concepts this guide uses

Sources

  1. interviewing.io: A Senior Engineer's Guide to the System Design Interview, part four (Pastebin and the Key Generation Service)
  2. Wikipedia: Universally unique identifier
  3. Pastebin.com: FAQ
  4. Amazon S3 User Guide: Downloading objects
  5. MDN: Cache-Control header
  6. Redis documentation: EXPIRE
  7. Redis cluster specification
  8. Amazon ElastiCache documentation: Caching strategies
  9. MongoDB manual: TTL Indexes
  10. MongoDB manual: Replication and Replica Set Fundamentals
  11. Cloudflare Cache docs: Purge by single-file
  12. Educative: Pastebin System Design
  13. System Design School: Design Pastebin
Sign in to report a mistake