Consistency, CAP and PACELC
Once data lives on several machines, "what does a read return?" stops having an obvious answer. This module defines the consistency guarantees you can ask for, explains what the famous CAP theorem really says (and doesn't), adds the everyday latency trade-off that PACELC names, and shows how systems reconcile copies that disagreed. The practical outcome is a habit: choose consistency per feature, not per system.
1. The idea
First: why distributed systems are hard
On one machine, a function call either returns or throws. Across a network there's a third outcome: nothing comes back, and you can't tell why. Every hard topic in this level (replication, consensus, transactions across services, exactly-once delivery) is a way of living with that.
A timeout tells you only that you didn't hear back. The request may never have arrived, may have been processed with the reply lost, or may still be running. Retrying is safe only if doing the operation twice is harmless (idempotency, Module 24).
Three things you can't trust
- The network: messages are lost, delayed, duplicated or reordered, and partitions cut machines off from each other while both keep running.
- Clocks: every machine's clock is a little wrong, and can jump. You can't order events across machines by timestamp alone (Module 20).
- Pauses: a process can freeze for seconds (garbage collection, a VM migration, a swapped-out page) and then carry on as if no time passed, still believing it's the leader.
What that forces on every design
- Partial failure: some parts work while others don't. Design each call for "slow", "failed" and "unknown".
- No shared view: each node only knows what messages told it. Agreement needs a protocol (quorums, consensus, Module 21).
- Trade-offs, not fixes: waiting for more replicas buys consistency and costs latency and availability. That's what CAP and PACELC below describe.
| The eight fallacies of distributed computing | What to do instead |
|---|---|
| The network is reliable | Timeouts, retries with backoff, idempotency keys |
| Latency is zero | Fewer round trips: batch, cache, avoid chains of calls |
| Bandwidth is infinite | Compress, paginate, send deltas, keep big data near compute |
| The network is secure | Encrypt and authenticate every hop (mTLS, Module 26) |
| Topology doesn't change | Service discovery and health checks, never hard-coded addresses |
| There is one administrator | Expect other teams' changes, versions and outages |
| Transport cost is zero | Count serialization CPU and cross-zone or cross-region transfer fees |
| The network is homogeneous | Standard protocols and formats; test on slow and lossy links |
The first seven fallacies were collected by L. Peter Deutsch and colleagues at Sun Microsystems in the 1990s; James Gosling added the eighth.
Consistency models, strongest to weakest
| Model | What readers are promised | Example need |
|---|---|---|
| Linearizable (strong) | The system behaves like one copy: once a write completes, every later read, anywhere, sees it. | Account balance, seat ownership, a lock, "is this username taken?" |
| Sequential | Everyone sees all writes in the same order, though perhaps late. | A replicated log everyone must agree on |
| Causal | If one write could have influenced another, everyone sees them in that order. Unrelated writes may appear in different orders. | A reply never shows before the comment it answers |
| Read-your-writes | You always see your own changes; others may see them later. | Editing your profile |
| Monotonic reads | Once you've seen a value, you never see an older one. | Refreshing a feed doesn't make posts vanish |
| Eventual | If writes stop, all copies converge. No promise about when, or what you read meanwhile. | Like counts, view counts, recommendations, DNS |
Note: this "consistency" is about replicas agreeing. ACID's "C" (Module 11) is about constraints holding. Same word, different ideas.
CAP, stated properly
In a distributed system, a network partition (some nodes can't reach others) will happen sooner or later. When it does, each side must choose:
- Consistency: refuse requests that might return stale data or create conflicts, until the partition heals; or
- Availability: keep answering on both sides, accepting that copies will disagree for a while.
Partition tolerance isn't optional (networks fail), so "CA" isn't a real choice for a distributed system. CAP is really "when partitioned, C or A?". Most of the time there is no partition, and that's where PACELC comes in.
PACELC: the everyday trade-off
If Partition, choose A or C; Else, choose Latency or Consistency. Even with a healthy network, keeping replicas strongly consistent means waiting for other replicas (often in other zones or regions) on every write or read. Lower latency means answering from one nearby copy and accepting staleness. This trade-off applies every millisecond, not just during rare outages.
2. How it works
Press Play: a three-node cluster, a network partition, and the two ways to respond.
Resolving conflicts after a partition
- Last write wins (LWW): keep the version with the latest timestamp, discard the other. Simple, but it silently loses data, and clocks on different machines disagree (Module 20).
- Keep both and merge: store both versions ("siblings") and merge with domain logic: union of cart items, max of a high score. Amazon's Dynamo paper famously merged shopping carts this way.
- CRDTs (conflict-free replicated data types): counters, sets, maps and text designed so concurrent updates always merge to the same result, without coordination. Used in collaborative editing and offline-first apps (Design D19).
- Avoid conflicts: route all writes for a given record to one leader (per user, per region), so concurrent writes to it can't happen.
ACID vs BASE
Two acronyms describe the two ends of this trade-off. ACID is what a traditional relational database promises for each transaction. BASE was coined, half as a joke, for systems that give some of that up to stay available and spread out over many machines.
| ACID | BASE |
|---|---|
| Atomic: all of a transaction's changes happen, or none do | Basically available: the system answers, even when some parts are failing or cut off |
| Consistent: the data's rules (constraints) hold before and after | Soft state: copies may differ for a while, even with no new writes arriving |
| Isolated: concurrent transactions don't see each other's half-done work (Module 17) | Eventually consistent: once writes stop, all copies converge to the same value |
| Durable: once committed, it survives crashes |
The same moment in each style:
It's a spectrum, not a binary choice, and most real systems mix the two: ACID for money, stock and anything with an invariant; BASE for counters, feeds, carts and analytics. Many databases offer both, per operation: DynamoDB and Cassandra are eventually consistent by default but offer transactions or conditional writes for the operations that need them, while Spanner and CockroachDB give ACID across regions at the cost of a coordination round-trip on each write.
3. Try it yourself
Partition a shopping cart
Two users share a cart. One talks to the majority side (nodes A and B), one to the minority side (node C). Cut the network, add items on both sides, then heal it, under each strategy.
Consistency or availability, per feature?
4. Trade-offs
Choose per feature
A ticketing site needs consistency when assigning a seat and availability when browsing events. A dating app needs consistency for matches and availability for profiles. Say it this way in designs.
Strong costs latency
Linearizable reads and writes need a majority of replicas to agree. Across zones that's a millisecond or two; across continents, tens or hundreds of milliseconds per operation.
Eventual needs a plan for conflicts
If both sides accept writes, something must decide the result later. "Last write wins" is a decision to lose data; make it on purpose.
Weaker guarantees, better tools
Read-your-writes and causal consistency fix most user-visible oddities at a fraction of strong consistency's cost.
5. In real systems
| System | Default | Can be tuned to |
|---|---|---|
| PostgreSQL / MySQL (single primary) | Strong on the primary; replicas lag | Synchronous replicas; read from the primary for freshness |
| DynamoDB | Eventually consistent reads | Strongly consistent reads (ConsistentRead) within a region, at higher cost; transactions |
| Cassandra / ScyllaDB | Per query; often LOCAL_QUORUM | ONE for speed, QUORUM or ALL for freshness (Module 14) |
| MongoDB | Reads and writes on the primary | Read and write concerns from "local" to "majority" and "linearizable" |
| Spanner, CockroachDB | Strong (serializable, externally consistent) across regions | Stale "follower reads" for lower latency |
| etcd, ZooKeeper | Linearizable writes via consensus | Used precisely because they choose C (Module 21) |
The choices up close
would it be catastrophic if a user briefly saw stale or conflicting data here?Yes → consistency; no → availability. Then say it per feature (
consistency for bookings, availability for browsing) and let it drive the database and replication choices.
Prioritise consistency
Situational · when stale data does harmDuring a partition, refuse (or delay) requests rather than risk a stale read or a conflicting write.
How it works
Writes and fresh reads go through one source of truth: a single primary database, a majority quorum, or a consensus system. Nodes that can't confirm with that source return an error or wait.
Status today
Single-primary PostgreSQL and MySQL, Spanner and CockroachDB, DynamoDB's strongly consistent reads and transactions, etcd and ZooKeeper.
Use it when
- Seat and ticket booking, the last item in stock
- Payments, balances, trading order books
- Uniqueness: usernames, locks, leader election
Skip it when
- Feeds, profiles, counts, recommendations: brief staleness there is invisible
In the interview
Booking a seat must never double-sell, so it's consistency first: one primary database with a conditional write. Users may see an error during a partition, which beats selling a seat twice.
- Choosing consistency for the whole system, including browsing, so a partition takes everything down.
- Saying "strong consistency" without the mechanism (which database, which write, which lock).
Prioritise availability
The default for most featuresKeep answering on every side of a partition, accepting that copies disagree for a while and converge later (eventual consistency).
How it works
Reads come from replicas and caches; writes may be accepted in several places and reconciled later (last write wins, merges, CRDTs). Change data capture keeps derived copies roughly up to date.
Status today
Cassandra and DynamoDB by default, read replicas and caches everywhere, DNS. Most user-facing reads in large products work this way.
Use it when
- Social feeds, profiles, likes and views
- Product descriptions, business hours, catalogue pages
- Anything where "a few seconds old" is fine
Skip it when
- Money, inventory, bookings, uniqueness
In the interview
Would it be catastrophic if a user briefly saw stale data? For the feed, no, so availability: replicas, caches and eventual consistency.
- "Eventual consistency" for a balance or a seat.
- Not saying how conflicts resolve: last write wins silently drops one of two concurrent updates.
Mix per feature
The senior answerCAP isn't a choice for the whole system. Decide per feature, early, in the non-functional requirements.
How it works
- Ticketmaster: consistency for booking seats, availability for browsing events.
- Tinder: consistency for creating a match, availability for viewing profiles.
- A bank: consistency for transfers, availability for the transaction history page.
Status today
How real products are built: one transactional source of truth for the critical writes, replicas and caches for everything else.
Use it when
- Every design with both critical writes and heavy browsing (most of them)
Skip it when
- Never: even a simple design benefits from saying which features need which
In the interview
I'll prioritise consistency for bookings and availability for browsing.
One sentence in the non-functional requirements, then let it drive the database and replication choices.
- Reciting CAP's definition instead of applying it to the features in front of you.
- Claiming "CA" for a distributed system: partitions aren't optional.
PACELC
Situational · a senior-level extensionIf there's a Partition, choose Availability or Consistency; Else, choose Latency or Consistency.
How it works
Even with a healthy network, strong consistency means waiting for other replicas (often in other zones or regions) on every write or read. Lower latency means answering from one nearby copy and accepting staleness. That trade-off is paid every millisecond, not just during rare outages.
Status today
A widely cited refinement of CAP (Abadi, 2010); databases are often described as PA/EL (Cassandra, DynamoDB defaults) or PC/EC (Spanner, single-primary SQL).
Use it when
- Multi-region designs, where cross-region consistency costs 50–150 ms per write
- When the interviewer probes latency vs freshness
Skip it when
- Simple single-region designs: CAP per feature is enough
In the interview
If latency comes up: Even without partitions, synchronous cross-region writes add about 100 ms, so only the ledger pays that; everything else reads locally.
- Name-dropping PACELC without connecting it to a latency number in your design.
Session guarantees and the levels in between
The default answer for user-facing dataMost user-facing features don't need linearizability; they need each user's own view to make sense. Four session guarantees (Terry and others, 1994) give exactly that, per client, and databases sell them as levels between strong and eventual.
How it works
| Guarantee | Promise, within one client's session | Bug it prevents |
|---|---|---|
| Read your writes | After you write, your reads include that write | You change your name and the page still shows the old one |
| Monotonic reads | You never see an older state after a newer one | A comment appears, then vanishes on refresh |
| Monotonic writes | Your writes are applied everywhere in the order you made them | "Set password" then "log out other devices" applied backwards |
| Writes follow reads | A write you make after reading something is ordered after what you read | Your reply shows up on a replica that doesn't have the post yet |
- How they're built: the client carries a session token, the highest version (log position) it has written or read. A replica serves the read only if it has caught up to that token; otherwise the read waits, retries on another replica, or goes to the leader. Sticky routing to one replica is the cheap version.
- Two more levels you'll meet: consistent prefix (you may be behind, but you see writes in an order that really happened: never an answer before its question) and bounded staleness (you may be behind, but by at most K versions or T seconds).
- Azure Cosmos DB names all five: strong, bounded staleness, session, consistent prefix, eventual. Its docs call session the most widely used level. Bounded staleness there has minimums of 100,000 writes or 300 seconds for multi-region accounts; if a region lags further, writes to that partition are throttled.
- Elsewhere: DynamoDB offers eventually consistent or strongly consistent reads per request; Spanner offers strong reads or stale reads with a bound you pick (for example, data at most 10 seconds old, served by the nearest replica); MongoDB offers causally consistent sessions.
Status today (checked October 2026)
Session-style consistency is what most large systems actually ship for user data. Checked against the Azure Cosmos DB consistency-levels documentation (October 2026).
Use it when
- Profiles, settings, carts, comments: anything a user writes and then looks at
- Read replicas or multiple regions where full linearizability costs too much latency
Skip it when
- Uniqueness, money, locks and stock counts: those need linearizable operations (Module 17, 21)
- Data nobody reads right after writing it, such as analytics events
In the interview
Feeds can be eventually consistent, but a user must see their own post immediately. I'd give reads a session token, the log position of the user's last write, and only serve them from a replica that has caught up to it.
- "Eventual consistency" with no plan for a user who reloads right after writing.
- Claiming read-your-writes across devices when the session token lives in one browser.
- Choosing strong consistency everywhere because the weaker levels weren't considered.
6. Remember it
How this hook works: what "refuse or guess, wait or guess" means
What the slogan means, phrase by phrase
- "When the line is cut"
- A network partition: the P in CAP and PACELC.
- "refuse or guess"
- Refuse = choose consistency (the minority side returns errors). Guess = choose availability (answer from local data that may be stale, accept writes that may conflict).
- "when it isn't, wait or guess"
- The E in PACELC: wait for other replicas (consistency, more latency) or answer from the nearest copy (latency, maybe stale).
Why this picture fits: it's the two bank branches with a cut phone line, plus the everyday version: even when the line works, phoning the other branch for every withdrawal takes time.
The story → the concept
| Two bank branches | Distributed data |
|---|---|
| The phone line between them | The network between replicas |
| The line is cut | A network partition |
| "Sorry, come back later" | Choosing consistency (CP) |
| Paying out from local records | Choosing availability (AP) |
| Reconciling the ledgers afterwards | Conflict resolution: LWW, merge, CRDTs |
| Phoning the other branch for every withdrawal | Synchronous, strongly consistent operations (latency) |
Watch it on an example
- A ticketing site's EU and US regions lose contact during a big sale.
- Seat assignment refuses on the minority side: some users see "try again shortly". No seat is sold twice.
- Event pages and search keep working everywhere from local replicas: maybe a description is a minute old.
- On a normal day, browsing reads the nearest replica (latency) while checkout waits for the primary (consistency).
The trap it saves you from
Declaring a whole system "CP" or "AP". Real designs pick per operation, and most of the cost shows up as latency on healthy days, not during partitions.
It still works when…
Mobile apps work offline: the phone is a partitioned replica that chooses availability, and syncs (with merges or CRDTs) when it reconnects.
7. Common mistakes
8. Design exercise (25 minutes)
- Classify the features of a food delivery app: menu browsing, cart, payment, order status, courier location, restaurant ratings. For each: the consistency model it needs and what users would see during a partition.
- Resolve a conflict: a note-taking app lets two offline devices edit the same note. Compare LWW, keeping both versions, and a CRDT. Which would you ship?
- Price the latency: users are in Europe and the US; strongly consistent writes need agreement across both. Estimate the extra latency per write (Module 02's round-trip numbers).
- Try the simulator with each strategy and write one line about what each one costs.
- Write it down: the deciding question, and the PACELC sentence.
9. Self-check quiz
What does the CAP theorem actually force you to choose?
What does the "ELC" in PACELC describe?
Which guarantee ensures a reply never appears before the comment it answers?
What's the main danger of last-write-wins?
Designing a ticketing site. What's the strongest CAP statement?
A call to the payment service times out. What do you know?
A user edits a comment, refreshes, and sees the old text. Which guarantee is missing?
How does a replica enforce read-your-writes with a session token?
10. Go deeper
| Resource | Time | Why |
|---|---|---|
| Designing Data-Intensive Applications, chapter 9 (Consistency and Consensus) | 2 h | Linearizability, ordering and CAP, explained carefully and correctly. |
| Consistency models (Jepsen) | 30 min | A clickable map of consistency models and how they relate, from the team that tests databases' claims. |
| Consistency Tradeoffs in Modern Distributed Database System Design (Daniel Abadi) | 30 min | The short paper that introduced PACELC. |
| crdt.tech | 30 min | An introduction to CRDTs, with links to papers and libraries such as Automerge and Yjs. |