Idempotency keys
A unique value the client attaches to a state-changing request so the server can recognize a retry and return the first result instead of doing the work twice. It is what makes retrying a payment safe.
What it is
An idempotency key is a unique value the client generates for one logical operation and sends with the request, usually in an Idempotency-Key header. The server keeps the key and uses it to recognize a later retry of that same operation. The need comes from an ambiguous failure: the client sends a POST, the connection drops, and it has no way to tell whether the server created the resource, is still working on it, or did not receive the request at all. HTTP already treats GET, PUT and DELETE as idempotent methods, so repeating one is meant to have the effect of a single call; POST and PATCH carry no such promise, and a repeated POST can create a second resource or apply an update twice. The key gives a POST the same retry safety by letting the server tell "the same operation again" apart from "a new operation".
What the server does with a repeat varies by API, and the detail matters. In Stripe's API v1, once an endpoint starts executing, the response status and body are kept under the key, and a retry with that key receives them again, even when the original was a 500. It compares the incoming parameters with the original's and returns an error when they differ. If the first request failed validation or collided with a concurrent request, nothing is saved and the retry runs fresh. API v2 behaves differently: a retry doesn't repeat work that succeeded; failed or partial work is retried to completion where possible and the response is updated, and when a retry can't succeed the API returns an error that says why. Many AWS operations model the same idea as a ClientToken field; the SDK and CLI fill one in when the caller doesn't and reuse it on their own retries, so the calling code sees a single response and no retries.
| Case | What the server does | Response |
|---|---|---|
New key | Record the key, do the work, store the result against the key | The normal result |
Key seen, work finished, same body | Treat it as a retry: answer as if the completed request had just succeeded | The saved status and body |
Key seen, first attempt still running | Don't start the work a second time |
|
Key seen, different body | Refuse: the key was reused for a different operation |
|
Pick the key on the client with enough randomness to avoid collisions: a version 4 UUID or a similar random identifier is the usual recommendation, and an API may cap their length (Stripe's limit is 255 characters). Don't build it from personal data such as an email address. Scope it to the caller: a store keyed on the pair of caller identity and key lets two customers use the same string without colliding, and stops one caller from replaying another's response. Keys are not a permanent archive. Stripe prunes v1 keys once they are at least 24 hours old and treats a key reused after that as a new request; one published retention rule for EC2 instances is the lifetime of the resource plus a grace period in which a late retry could still arrive. The IETF draft asks a server to publish whatever expiry policy it chooses.
The record of the key and the work it protects need to commit together. If the key is written as complete before the work commits and the work then fails, a retry can be told the operation is done when it isn't; if the work commits and the key write fails, a retry does the work again. Amazon's guidance is that recording the token and the mutations form one ACID operation. When the operation calls another system (a card network, an email provider, a message broker), that call sits outside your transaction and won't roll back with it, so one practical design groups the local writes between such calls into separate transactions, records a recovery point on the key after each, and lets a retry resume from the last point reached instead of from the start. Passing your own key to the downstream service, when it accepts one, makes that call safe to repeat as well.
When to use it
Use a key on a request that creates or changes something, that a client might send twice, and that has no natural protection of its own: a payment, an order, a transfer, a resource launch, an action that triggers an email. The retry may be deliberate (the client saw a timeout) or invisible (a client library retries on its own), and both need the same key for the repeat to be harmless. Pair the retry with exponential backoff and random jitter so a struggling server isn't hit by all of its clients at the same moment; the Stripe Ruby library, for example, retries with a key, growing waits and jitter.
The pattern fits inside a system as much as at the public edge. A queue that delivers at least once hands the consumer duplicates, and a key stored with the processed result is what lets the consumer skip the second copy; see Delivery semantics. Calls into your own infrastructure count too: a write to a message broker such as Kafka is a fallible call outside your database transaction and deserves the same care as a call to a payment provider.
When not to
Skip the key when the method is already idempotent. A PUT that carries the full desired state, or a DELETE of a named resource, has the same effect repeated as once, and one widely used payment API documents that a key on GET or DELETE has no effect. Skip it when a natural unique constraint already does the job: creating a DNS record that is named by its hostname, or inserting a row whose business identifier is unique, can treat a second attempt as a success with no extra bookkeeping.
Don't derive the key from the request's contents. Hashing the parameters looks simpler, but two deliberate, identical requests (two instances of the same shape, two identical gifts) would be treated as one, which is why the guidance from large API providers is a caller-supplied identifier. Don't mistake the key for a lock on the resource either: it dedupes repeats of one operation, not two different operations that touch the same row; for that, see Optimistic and pessimistic locking. And where an occasional duplicate is acceptable and the operation is cheap (an approximate view counter, a "seen" flag), the key store, the lookup and the lock can cost more than the duplicate would.
Compared
| Approach | How a repeat is recognized | Right choice when | Cost or limit |
|---|---|---|---|
Idempotent method ( | The second call rewrites the same state | The client knows the final state and the resource has a stable address | Doesn't fit "create one more" or "charge once" |
Natural unique constraint | A unique index refuses the second insert and the handler reports success | The operation has a real business identifier (order number, hostname) | Needs such an identifier, and the handler has to map the duplicate error to a success response itself |
Hash of the parameters | Same caller, same inputs, within a window | Identical inputs are not a valid second request | Merges two deliberate identical requests into one |
Client-supplied idempotency key | Same caller, same key |
| A key store with a lifetime, a parameter check and handling for in-flight repeats |
In a picture
How to say it in an interview
The common mistake
Quiz
Questions that use this
Related
Sources
- Stripe API reference: Idempotent requests
- IETF Internet-Draft: The Idempotency-Key HTTP Header Field (draft-ietf-httpapi-idempotency-key-header-09)
- Amazon Builders' Library: Making retries safe with idempotent APIs
- Stripe blog: Designing predictable APIs with idempotency
- brandur.org: Implementing Stripe-like Idempotency Keys in Postgres