How to design Dropbox
The first sections are free. The deep dives, level expectations and quiz are part of Pro.
On this page
- The problem
- Clarifying questions
- The requirements menu
- Non-functional requirements
- Estimates
- Core entities
- The API
- The high-level design
- What breaks first
- The deep dives
- Entity fields
- Uploading a 50 GB file
- Keeping devices in sync
- Sharing and signed URLs
- Variants
- What each level is expected to show
- The 30-second answer
- Quiz
The problem
A file storage service like Dropbox keeps each file's bytes in blob storage, keeps what it knows about the file (name, size, owner, which chunks make it up) in a database, and runs a small agent on every device that notices local edits and learns about remote ones. The answer turns on one separation: the servers you write handle metadata and permissions, while the bytes travel straight between the device and storage in chunks. Once that holds, a dropped 50 GB upload costs one chunk, a changed file costs the chunks that changed, and a download is a signed link the CDN can check without calling you.
Clarifying questions
| Ask | Why it matters | A typical answer |
|---|---|---|
How large can a single file be, and over what kind of connection do people upload? | The largest file decides whether one HTTP request can carry it. If the answer is tens of gigabytes over home internet, the upload path must be chunked and resumable from the first diagram. | Files up to 50 GB, uploaded over connections around 100 Mbps. At that speed a 50 GB file takes about 4,000 seconds, more than an hour. |
How many people use it, and how often do they upload? | It tells you how much of the load is control-plane calls (small, many) against bytes (few, huge), and whether the file service needs anything more than ordinary horizontal scaling. | About 10 million daily users of 50 million sign-ups. Assume about two uploads per person a day of roughly 500 KB on average, which is about 240 upload requests a second, 480 at peak. |
Are files edited in place, or replaced whole from a device? | Collaborative editing inside the browser is a different system (operational transforms or CRDTs). Treating files as opaque bytes that a device rewrites keeps the problem to storage and sync. | Opaque files. People edit them with their own applications; the agent on the device notices the saved file changed and uploads the difference. |
How quickly must a change made on one device appear on another? | Seconds against milliseconds decides between a push channel and plain polling, and whether metadata can favor availability over strict consistency. | A few seconds is fine, and a few seconds before an upload shows in another region is acceptable. Availability wins over consistency for metadata. |
Who do people share with: named users, or anyone holding a link? | Sharing with named users needs a table the file service can check; a public link is a bearer token and brings abuse and traffic limits with it. | Sharing with named users is in scope, and seeing what was shared with you. Keep in mind that one free account's links can serve up to 100,000 downloads a day, so downloads must not pass through your servers. |
What should happen when two devices change one file before either has synced? | There is no merge for opaque bytes, so you must pick a policy: one version wins, or both are kept. Saying which and why is part of the sync discussion. | Your call. Be ready to defend it: silently keeping the last writer loses someone's work, keeping both versions makes the person decide. |
Do we need version history, quotas, previews or virus scanning? | Each is a real feature in the product, and each is a separate system. Naming them and parking them keeps the round on uploads, sync and sharing. | Previews, quotas, in-browser editing and virus scanning are out of scope. Version history is optional: keep old chunk sets around if you have time. |
Non-functional requirements
| Requirement | What it forces |
|---|---|
Availability over consistency: a few seconds before an upload shows in another region is fine | Metadata can live in a database replicated asynchronously, and the change feed to other devices can lag. It also means a device may briefly hold a version the server has already superseded, so the sync design needs a rule for what happens when it uploads anyway. |
Files up to 50 GB | No single request carries a file: an upload is split into chunks the client sends straight to blob storage, each one retried and resumed on its own. Nothing on the path, gateway or service, ever holds a whole file in memory. |
No file is lost or corrupted | The service marks a file as uploaded only after storage confirms each chunk, not when the client says so. Chunk and file fingerprints travel with the upload, and storage or the service recomputes and compares them so a corrupted part is caught; blob storage is trusted for durability of the bytes. |
Low latency for upload, download and sync | Chunks upload in parallel, downloads come from a CDN edge near the device, and a remote change is announced on a connection the device keeps open, instead of waiting for the next poll. The file service does one metadata round trip per call and never streams bytes. |
Estimates
| Quantity | Working | Result | What it sizes |
|---|---|---|---|
Time to upload one 50 GB file at 100 Mbps | 50 GB × 8 = 400,000 megabits; 400,000 Mb ÷ 100 Mbps = 4,000 seconds ≈ 1.1 hours | About 4,000 seconds, a little over an hour | Whether a single request can carry the largest file. More than an hour on one connection is likely to meet a timeout or a dropped Wi-Fi link somewhere along the way, so uploads must be chunked and resumable. |
Chunks in a 50 GB file | 50 GB ÷ 5 MB = 10,000 chunks; 50 GB ÷ 10 MB = 5,000 chunks; 50 GB ÷ 4 MB = 12,500 chunks. S3 allows parts of 5 MiB to 5 GiB and at most 10,000 of them per upload | 5,000 to 12,500 chunks, depending on chunk size | The chunk size: S3 caps a multipart upload at 10,000 parts, and each part other than the last must be at least 5 MiB, a little more than 5 MB. So 5 MB chunks fall under the minimum and would need all 10,000 part numbers for 50 GB; 4 MB chunks miss both limits; a 10 MB part clears both with room to spare. |
What a drop at 49 GB costs, with and without chunks | Whole file: the 49 GB already sent (49 GB × 8 ÷ 100 Mbps = 3,920 seconds ≈ 65 minutes) is wasted, and a restart from the first byte sends all 50 GB again. Chunked: one 10 MB chunk × 8 ÷ 100 Mbps = 0.8 seconds | About 65 minutes lost against under a second | The value of resumable uploads: with chunks the client re-sends the one part in flight, not the hour of work behind it. |
Upload requests a second | 10 million daily users × 2 uploads a day = 20 million a day; 20 million ÷ 86,400 seconds ≈ 231, call it about 240; peak taken as twice the average, 480 | About 240 a second on average, 480 at peak | The control-plane load on the file service: a few hundred small metadata calls a second, which ordinary stateless servers behind a load balancer handle without a queue in front. |
Bytes uploaded a day | 20 million uploads × 500 KB = 10 TB a day; × 365 ≈ 3.65 PB a year | About 10 TB a day, about 3.65 PB a year of new blobs | Blob storage growth, and a reminder that the 500 PB allocated as free quota is far above what is actually written; storage is sized on real uploads, not on quotas. |
One free account's shared-link traffic | 100,000 downloads ÷ 86,400 seconds ≈ 1.2 downloads a second; 20 GB ÷ 100,000 ≈ 200 KB per download on average | About 1.2 downloads a second and 20 GB a day from a single account | Why downloads bypass the file service: one account can push 20 GB a day through its links, and a popular file served from your servers would be the first thing to fall over. The CDN and blob storage carry the bytes; the file service only signs. |
What one inserted byte costs with fixed chunks | A 50 GB file in 4 MB fixed chunks is 12,500 chunks; inserting a byte at the start shifts every boundary, so all 12,500 change: 50 GB re-uploaded, about 4,000 seconds. With content-defined chunks usually only a chunk or two around the edit changes: 4 to 8 MB, under a second | 50 GB against about 8 MB for the same edit | Why delta sync needs chunk boundaries chosen by content rather than by offset. |
A 50 GB download against a 5 minute link expiry | 5 minutes = 300 seconds; a 50 GB transfer on a 100 Mbps link ≈ 4,000 seconds, about 13 times longer than the link is valid | The transfer outlives the link many times over | What the expiry on a signed URL must mean: checked when the request starts, not for the whole transfer. A client that downloads in chunks should expect to ask for fresh links partway through. |
Core entities
| Entity | What it holds |
|---|---|
File metadata | One row per file: what it is called, who owns it, how big it is, which chunks make it up and whether the upload has finished. |
Chunk | One row per chunk of a file version: where it sits in the file, its hash, and whether storage has confirmed it. |
Shared file | One row per person a file was shared with, keyed so 'files shared with me' is a single lookup. |
User | An account. Devices hang off it, and ownership and shares point at it. |
Device | One installed agent. The change notifier addresses devices, not users, because each device has its own cursor into the change feed. |
The API
| Endpoint | Notes |
|---|---|
| The client sends name, size, mime type, the whole-file fingerprint and the list of chunk hashes. The service writes a metadata row with |
| The client reports a part landed. The service records it as reported, not verified: before completing the upload it lists the parts storage actually holds and compares. Idempotent, so a retried report after a timeout does no harm. |
| Checks that the caller owns the file or has a row in the shares table, then signs a short-lived download URL for the CDN. A caller with neither gets an error and no link. The expiry lives in the link, so a fresh call is needed for each download. |
| Inserts one shares row per recipient and emits a change event for each recipient's devices, so the file shows up in their 'shared with me' list without a refresh. Un-sharing deletes the row; a link already signed keeps working until it expires. |
| The catch-up call for sync. A device that was offline, or that missed a push, asks from the cursor it saved after its last successful sync and receives the file versions it has not seen, plus a new cursor. Paginate it, since a device that was offline for a long time may have many changes to fetch. |
The high-level design
Step 1 of 4
A file service and a metadata table
Start with the smallest thing that stores a file. The agent on a device calls a file service, which writes a row of metadata (name, size, owner, fingerprint) and, in this first version, also receives the bytes and writes them to disk. Downloads reverse it. This is enough to talk through the entities and the API, and it is wrong in a way the numbers make obvious in the next step.
What breaks first
The deep dives
Uploading a 50 GB file. Someone uploads a 50 GB video over home Wi-Fi and the connection drops at 49 GB. How do the bytes reach storage, what does the client do when it reconnects, and when does the file service decide the file is really there?
Keeping devices in sync. A person edits a large file on their laptop while their phone is in airplane mode. How does the laptop decide what to upload, how does the phone learn about it when it comes back, and what happens if both devices changed the file?
Sharing and signed URLs. Alice shares a file with Bob, Bob keeps the download link his client was given, and then Alice un-shares it. What can Bob still open, and what can a stranger Bob forwarded the link to open?
Concepts this guide uses
Sources
- Amazon S3 User Guide: Uploading and copying objects using multipart upload
- Amazon S3 User Guide: Amazon S3 multipart upload limits
- Amazon S3 User Guide: Sharing objects with presigned URLs
- Amazon CloudFront Developer Guide: Use signed URLs
- Dropbox Help: Why has my sharing activity been interrupted?
- Dropbox Help: Why am I getting conflicted copies while using the Dropbox badge?
- Dropbox Tech Blog: Rewriting the heart of our sync engine