Space-based architecture and caching
Almost every architecture eventually hits the same wall: the database. You can add web servers for a burst, but they all queue on one database. Space-based architecture removes the database from the request path entirely: each processing unit keeps the data it needs in memory, copies changes to its peers, and writes to the database later, in the background. It's how ticket sales, auctions and trading platforms survive extreme, unpredictable bursts. It also relies on caching, so this module covers the four caching topologies and the collisions that replicated caches cause.
1. The idea
- Processing units: copies of the application, each holding an in-memory data grid with the data it needs. Requests are served from memory.
- Virtualised middleware: a messaging grid routes requests to units, a data grid replicates changes between units, a processing grid coordinates requests that need several units, and a deployment manager starts and stops units as load changes.
- Data pumps send changes asynchronously (usually through a queue) to data writers that update the database. Data readers load data into units when they start cold.
The four caching topologies
| Topology | How it works | Good for | Watch out for |
|---|---|---|---|
| Single in-memory | Each instance caches on its own; nothing is shared | One instance, or data that never changes | Instances disagree once data changes |
| Distributed | A separate cache cluster (such as Redis) that every instance calls over the network | Large data, frequent updates, many instances | A network hop per read; the cache cluster is a dependency |
| Replicated | Every instance holds a full copy in its own memory; changes are copied to the others | Fast reads of small to medium data, few updates | Memory per instance; collisions on concurrent updates |
| Near-cache hybrid | A small local cache of the most-used items in front of a distributed cache | Hot items read very often | The local layer can be briefly stale |
How it rates
| Characteristic | Space-based | Event-driven |
|---|---|---|
| Partitioning | technical | technical |
| Quanta | 1 to many | 1 to many |
| Cost | ★★★★★ | ★★★★★ |
| Simplicity | ★★★★★ | ★★★★★ |
| Scalability | ★★★★★ | ★★★★★ |
| Elasticity | ★★★★★ | ★★★★★ |
| Fault tolerance | ★★★★★ | ★★★★★ |
| Evolvability | ★★★★★ | ★★★★★ |
| Deployability | ★★★★★ | ★★★★★ |
| Testability | ★★★★★ | ★★★★★ |
| Performance | ★★★★★ | ★★★★★ |
Our qualitative ratings, 1 to 5 stars, more is better (for cost, more stars = cheaper). They compare styles in general; your context can move any of them.
2. How it works
A burst without a database bottleneck
A data collision in a replicated cache
Replication takes time. If two units update the same item within that window, one update can overwrite the other. Watch two units sell seats for the same concert.
3. Try it yourself
How often will replicated caches collide?
Collisions grow with the number of units, the update rate and the replication delay, and shrink with the number of items being updated. Adjust them and see the expected collisions per hour.
Expected collisions per second ≈ N × (N − 1) × u² × delay ÷ items, for N units each making u updates a second spread evenly over the items. We checked the formula against a simulation; real updates are rarely even (popular items collide far more), so treat it as a lower bound.
4. Trade-offs
Elasticity vs consistency
Taking the database off the request path gives near-unlimited elasticity and speed. The database is now eventually consistent with memory, and concurrent updates in different units can collide.
Memory vs data size
Each unit must hold the data it needs in memory. That suits a bounded working set (this week's events, today's orders), not a decade of history.
Replicated vs distributed cache
Replicated caches read at memory speed but cost memory in every unit and collide on updates. Distributed caches hold one copy and avoid collisions, at the price of a network hop per read.
Power vs complexity
Space-based is one of the most complex styles to build and test: middleware, data pumps, readers and writers, and collision handling. Use it only where bursts truly demand it.
5. In real projects
Where it fits: concert and sports ticketing, online auctions, flash sales, betting and trading platforms: huge, sudden concurrency on a relatively small, hot data set. In-memory data grids such as Hazelcast, Apache Ignite and Oracle Coherence provide the grid pieces.
Choosing a caching topology comes down to four questions: how big is the data (replicated needs it to fit in every unit), how often does it change (frequent updates favour distributed), how many instances (more instances mean more collisions for replicated), and how fresh must reads be.
Caching and CAP. A replicated cache chooses availability: every unit answers from its own copy, even if it's briefly stale. A distributed cache with one copy per item is closer to consistency: everyone sees the same value, but a cache-node outage makes those items unavailable.
Say it in the interview
- "What is space-based architecture?" Processing units that keep their data in replicated in-memory grids, with the database updated asynchronously through data pumps. Removing the database from the request path gives extreme elasticity for bursts.
- "Replicated or distributed cache?" Replicated for small, mostly-read data where read speed matters; distributed for large or frequently updated data, or many instances. Updates to replicated caches can collide.
- "How do you handle the last-seat problem?" Partition so each seat or event is owned by one unit, or route updates for a key to its owner; otherwise accept and detect collisions. Pure replication can't prevent them.
- A replicated cache for data updated thousands of times a second.
- Space-based architecture for a steady, ordinary workload.
- No plan for what happens when the data pump's queue backs up.
6. Remember it
How this hook works: what "a fridge at every bar" means
What the slogan means, phrase by phrase
- "A fridge"
- An in-memory data grid inside each processing unit.
- "at every bar"
- Every unit has one, so opening more bars (units) adds capacity without queuing at a storeroom.
Why this picture fits: it explains the elasticity (no trip to the storeroom) and the risk (two bars selling the last bottle before they've phoned each other).
The story → the concept
| Festival bars | Space-based architecture |
|---|---|
| A bar with its own fridge | Processing unit with an in-memory grid |
| Bars phoning each other | Data grid replication |
| The overnight van to the storeroom | Data pump and data writer |
| Two bars selling the last bottle | A data collision |
The trap it saves you from
Scaling application servers for a burst while every one of them waits on the same database.
7. Common mistakes
8. Design exercise (30 minutes)
- Size the working set (10 min). For a ticketing site, estimate the data one event's sale needs in memory: seats, prices, holds. Does it fit in one unit?
- Pick a topology (10 min). For seat availability, prices and customer profiles, choose a caching topology each, and say why.
- Handle the last seat (10 min). Design how two units avoid selling the same seat: ownership by partition, routing by key, or detection and repair.
9. Self-check quiz
Why does space-based architecture handle bursts so well?
Doubling the number of units with a replicated cache (from 4 to 8) changes collisions how?
Large data, updated often, read by 30 instances. Which caching topology?
What does a data pump do?
In CAP terms, what does a replicated cache favour?
10. Go deeper
| Resource | Time | Why |
|---|---|---|
| Fundamentals of Software Architecture, 2nd edition (Richards and Ford) | book | The space-based chapter: processing units, virtualised middleware, data pumps and collisions. |
| Hazelcast: choosing a deployment option | 15 min | Embedded vs client-server in-memory grids in a real product. |
| Two Hard Things (Martin Fowler) | 5 min | Cache invalidation, and why it's famously hard. |