Running in several regions
Serving one system from several geographic regions at once: how a request finds the nearest healthy region, which regions may accept writes, and how long a change takes to show up elsewhere. The write path is the hard part.
What it is
A region is a geographic area in which a cloud provider runs infrastructure, split into zones; a multi-region system runs in two or more such regions at the same time. Teams do this for three reasons: reads with local latency for users who sit far from the original site, recovery when a whole region goes down, and keeping a subset of data stored in a chosen set of regions, which CockroachDB's v26.4 documentation calls data domiciling and supports with super regions.
Getting the request to a region. DNS can do the routing. With Amazon Route 53 latency records you create one record per region, and the resolver gets the region with the best measured latency among the ones you listed. Attach a health check and the record is used while its endpoint is healthy, so a dead region drops out of the answers. The record's TTL decides how fast clients follow that change: a longer TTL means resolvers keep the old answer longer, and Route 53 suggests 60 seconds or less for records tied to a health check.
| Shape | Who takes writes | What a product does | What it costs you |
|---|---|---|---|
One writer, regional read copies | The primary region; other regions can forward writes to it | Aurora Global Database: one primary cluster and up to 10 secondary regions that serve reads, replicated with latency typically under a second | Writes from far away cross the ocean; reads in a secondary can lag the primary |
Home region per record | The record's home region | CockroachDB regional-by-row tables: each row carries a home region, and the database optimizes access to the row from that region | Access from outside the home region is not what the table is optimized for |
Each region writes | Any region, for any record | DynamoDB global tables (version 2019.11.21) in the default eventual mode: each replica accepts writes and copies them to the others asynchronously | Concurrent edits to the same item collide; DynamoDB keeps the write with the latest internal timestamp |
Synchronous across regions | Any region, after a majority agrees | DynamoDB's strong consistency mode replicates a write to at least one other region before it succeeds, in exactly three regions, optionally two replicas plus a witness | Each write and strong read waits on the round trip between the regions you chose |
The gap after a write. With asynchronous copying, a write is acknowledged by one region and shipped to the others afterward. Both AWS products above describe the delay as typically within a second, and DynamoDB puts the recovery point objective of an eventual-mode table at the current replication delay, usually a few seconds depending on which regions you picked. Inside that window a reader in another region sees the previous value, or nothing if the record is new. DynamoDB's strongly consistent reads do not close the gap: they return the newest version for items last written in the same region and may return stale data for items written elsewhere. Transactions in that mode are atomic inside the region that ran them, and other regions may see part of a transaction before the rest.
Making IDs in several regions. When more than one region creates records, two regions must not mint the same key. In a last-writer-wins store the later write would replace the earlier one and both callers would believe they succeeded. Our advice: give each region a slice of the key space before it starts, with a region character in the key, a counter range reserved per region, or a few region bits inside a generated ID (see Unique ID generation). A key that names its birth region also tells a reader in another region where to go when the local copy has not arrived yet.
When to use it
Users on more than one continent, and read latency matters. Start with one writer region and read copies near the readers; Aurora Global Database is built for exactly this, with secondary clusters serving local reads and write forwarding so an application in a secondary region can still issue writes that the primary executes. If the design's reads far outnumber its writes, this shape buys most of the latency win with the least new behavior.
A region outage must not take the product down. You need a second region that can become the writer. Aurora separates a planned switchover, which moves the primary to a secondary region with no data loss, from a failover after an outage, where the unreplicated tail can be lost. CockroachDB makes the same choice explicit as a survival goal: a multi-region database survives a zone failure by default, and you opt into surviving a whole region, a setting the documentation says has a significant effect on performance. Say which goal you are paying for.
Data must stay in a jurisdiction, or most activity on a record comes from one place. Give each record a home region. In CockroachDB a regional-by-row table stores a home region on each row, and the database optimizes access to the row from that region. A home region on its own does not pin the data: a super region keeps all replicas of those rows inside the regions you allow, which is the tool for residency. CockroachDB also offers a global table locality, and suggests lowering the clock offset bound to 250 ms because that lowers the write latency of global tables.
Writes come from everywhere and a lost concurrent edit is acceptable. Then an active-active store with last-writer-wins, such as a DynamoDB global table in eventual mode, keeps writes local. Pair it with keys that embed the origin region, and read Conflict resolution before you accept the default merge rule.
When not to
One region with several zones may already cover the failure you must survive. Cloud providers divide a region into zones, and CockroachDB's default survival goal is zone failure rather than region failure. Spreading across zones inside one region covers the loss of one zone without adding cross-region lag to your reads or writes. Add a second region when a requirement names it: a latency target for distant users, a recovery plan that must survive a regional outage, or a residency rule.
You need multi-item transactions that hold across regions. In a DynamoDB global table with eventual consistency, a transaction is atomic in the region where it ran and is not shipped to other regions as a unit. If a balance and a ledger row must move together everywhere, keep that data in one writer region, or pick a database whose cross-region mode is documented to keep transactions atomic.
Each write must be visible to the next read anywhere, and your regions are far apart. The synchronous options make writes and strong reads wait on the round trip between regions, and AWS notes that a greater distance between the participating regions raises request latency. If that latency is not acceptable, a single writer region with a stated staleness window is usually the honest design. Each added region also widens what your deploys, monitoring and on-call must cover; a small team should name that cost before promising three regions.
Compared
| Shape | A read right after a remote write | Write latency | Pick it when |
|---|---|---|---|
Single region, several zones | No cross-region lag; what a read sees depends on the database's own read mode | Local | Users are near one place and a zone failure is the risk you must cover |
One writer, regional read copies | Lags the primary (Aurora documents typically under a second); stale or missing until the copy lands | Cross-region for users far from the primary | Reads dominate and a short, stated staleness window is fine |
Home region per record | Local for the owner; a remote visitor is routed home or waits for the copy | Local for the owner | Activity on a record clusters by geography; add replica placement limits (such as super regions) when data must stay in one place |
Each region writes, asynchronous copies | May see the old value; concurrent edits need a conflict rule (DynamoDB uses last writer wins) | Local | Writes come from everywhere and a lost concurrent edit is acceptable |
Synchronous across regions | A strongly consistent read sees the write | A cross-region round trip on each write | Correctness beats latency and the regions are close enough to afford the trip |
In a picture
Step 1 of 3
DNS picks the nearest healthy region
A latency record per region and a health check on each: the user is sent to the healthy region with the best latency, which is not always the closest. The application there writes to its local store.