Blob storage and presigned uploads
An object store holds files as blobs behind a key; a presigned URL lets a client upload or fetch one directly, within limits your server signs, so large bytes skip your API tier.
What it is
An object store keeps a file as an object: the bytes plus some metadata, addressed by a key that is unique within its bucket. Clients read and write objects over HTTP, so bucket plus key is what your application needs to record. Objects can be big: S3 takes up to 5 GB in a single PUT and up to 50 TB when the object is sent in parts. Your database keeps the key and the metadata (owner, size, content type, state) while the bytes stay in the store.
A presigned URL moves the authorization into the URL itself. Your server, which holds credentials, signs the bucket, the key, one HTTP method and an expiry, then hands the result to the client. Whoever holds that URL can make that one kind of request until it expires, with no account of their own. The URL can do no more than its signer: if the signing identity lacks permission to write that key, the upload fails, and a URL signed with temporary credentials stops working when those credentials expire, even if its own expiry is later. Google Cloud calls the same thing a signed URL; Azure uses a shared access signature (SAS), a token in the query string that scopes the resource, the permissions and the validity window.
| Provider | Upload token | Longest validity | Note |
|---|---|---|---|
Amazon S3 | Presigned URL (SigV4) | 7 days from the CLI or SDKs; 12 hours from the console | Shorter when the signing credentials are temporary |
Google Cloud Storage | Signed URL (V4) | 7 days | The holder needs no Google account |
Azure Blob Storage | Shared access signature | Set by the signer; near-term expiry is recommended | A user delegation SAS, signed with Entra credentials, is the recommended kind |
The upload has three steps, and the middle one is the one your servers don't see. First, the client asks your API to upload; the API checks that this user may add a file here, writes a metadata row in a pending state, and returns a signed URL for a key it chose. Second, the client sends the bytes to the store with that URL. Third, something confirms the bytes landed: the client calls back and the API checks that the object exists, or the store emits an object-created event to a queue or a function. S3 event notifications are designed for at-least-once delivery and usually arrive within seconds, though sometimes a minute or more, so the handler must tolerate a duplicate and the row may lag the upload a little. Then the row flips to ready. Treat anything written through a signed URL as unvalidated until your own code has checked it, since a client can send content other than what you expected.
A POST policy upload, the HTML form variant in the S3-compatible API family, embeds a signed policy in the form. The policy carries an expiration and a list of conditions the form fields must meet: an exact match on the bucket, a starts-with prefix on the key (so a user can write under their own folder and nowhere else), a prefix or exact value for Content-Type, and a content-length-range with a minimum and a maximum body size. If you need a hard bound on upload size, that condition is the way to put it into the signature. When the uploader is a page in a browser, the bucket also needs a CORS configuration that lists your origin and the allowed methods before the page can call it.
When to use it
Use the store plus a signed URL when users send files of meaningful size (photos, video, documents, exports) and your API would otherwise spend its bandwidth and memory relaying bytes. The store scales the byte path; your servers keep the small, fast work: authorization, key generation, metadata, confirmation. Mobile and browser clients fit well here, because you don't want to ship cloud credentials inside them and a signed URL gives them time-limited access without any.
Use the same trick for downloads of private files: a signed GET with a short expiry lets the browser fetch straight from the store, and for public or widely shared files a CDN in front of the store does better (see CDNs and edge caching). At 100 MB or more, or on connections that drop, send the object in parts: parts travel independently and in any order, and a failed part is retried on its own; that is covered in Chunked and resumable uploads. Parts of an upload that isn't completed still occupy storage and are billed, so set a lifecycle rule that aborts incomplete multipart uploads after a few days.
When not to
Keep small blobs that are read together with their row (a tiny thumbnail, a settings document of a few kilobytes) in the database or inline in the response; a second round trip and a second system to keep consistent cost more than they save. Relay the upload through your API when the content must be inspected or transformed before it may exist at all (a strict compliance rule, a synchronous conversion), when traffic is low enough that the relay is cheap, or when you want one place for validation logic and don't yet have a scale problem. Accepting first and validating afterward also works, but the pending state and the cleanup are extra machinery.
Skip signed URLs between your own services: a backend with its own cloud identity calls the store directly through the SDK. And don't lean on a signed URL for access you may need to withdraw quickly: Azure's guidance is to have a revocation plan before you issue one, and a stored access policy, which a service SAS can be tied to, is the lever it offers for pulling permissions back. A short expiry and a narrow key are the practical limits.
Compared
| Approach | Path of the bytes | What you control | When it's the right choice |
|---|---|---|---|
Upload through your API | Client to API to store | All of it: you see the bytes before they're stored | Small files, low traffic, or content that must be checked or transformed before it may be stored |
Presigned PUT | Client straight to store | Key, method, expiry, signed headers | A trusted client app uploading one file at a time, where size needs no hard bound |
POST policy (HTML form) | Client straight to store, via a form | Plus a size range, a key prefix, a content type match | Browser uploads where you need a size bound or a plain HTML form |
Multipart upload | Client to store, in parts | Parts sent independently, in any order, each retried on its own | Objects of 100 MB or more, or flaky connections; more bookkeeping and a cleanup rule |
In a picture
How to say it in an interview
The common mistake
Quiz
Questions that use this
Related
Sources
- Amazon S3 User Guide: What is Amazon S3?
- Amazon S3 User Guide: Uploading objects
- Amazon S3 User Guide: Download and upload objects with presigned URLs
- Oracle Cloud Infrastructure documentation: Uploads Using POST with Amazon S3 Compatibility API for Object Storage
- Amazon S3 User Guide: Amazon S3 Event Notifications
- Amazon S3 User Guide: Using cross-origin resource sharing (CORS)
- Amazon S3 User Guide: Uploading and copying objects using multipart upload
- Google Cloud Storage documentation: Signed URLs
- Azure Storage documentation: Grant limited access to Azure Storage resources using shared access signatures (SAS)