Space-based

Space-based architecture takes the database off the request path so that a sudden surge can't overwhelm it: copies of the app serve everything from replicated memory, and the database catches up in batches. This page builds Pedal's festival surge that way, survives a unit crash, and measures what the write-behind window puts at risk.

1. In one picture

Space-based architecture removes the database from the request path. Each processing unit is a copy of the application that keeps the working data in memory, in a data grid replicated between the units. Writes go to memory (and to the other units), and a data writer saves them to the database later, in batches ("write-behind"). A data reader loads the grid from the database when the system starts.

It's built for sudden, extreme load on the same hot data: ticket on-sales, auctions, flash sales, a city-wide surge of rides. Because units share nothing but the grid, more can start in seconds. The name comes from tuple spaces, the shared-memory model it grew from; Architecture module 15 has the architect's view.

2. Pedal built this way

A festival ends and 20,000 people want bikes in the same half hour. Follow two unlocks, a database flush, and a unit crashing mid-surge.

3. The rules

4. How it goes wrong

5. The code

Processing units that keep rides in a replicated in-memory grid, a write-behind queue, and a data writer that saves it to the database in one batch. When unit-2 crashes, unit-3 starts by copying the grid from unit-1.

from __future__ import annotations


class Database:                                   # slow, durable, updated later
    def __init__(self) -> None:
        self.rides: dict[str, dict] = {}


class ProcessingUnit:
    """A copy of the application holding its working data in memory, kept in step with its peers."""

    def __init__(self, name: str, peers: list["ProcessingUnit"], write_behind: list[tuple[str, dict]]) -> None:
        self.name, self.peers, self.queue = name, peers, write_behind
        self.grid: dict[str, dict] = {}
        for p in peers:                           # a new unit copies the grid from a running peer
            self.grid = {k: dict(v) for k, v in p.grid.items()}
            break
        peers.append(self)

    def put(self, key: str, value: dict) -> None:
        for p in self.peers:                      # replicate to every unit's memory
            p.grid[key] = dict(value)
        self.queue.append((key, dict(value)))     # the database hears about it later

    def unlock(self, rider: str, bike: str) -> str:
        ride_id = f"R{len(self.grid) + 1}"
        self.put(ride_id, {"rider": rider, "bike": bike, "open": True})
        return ride_id


def flush(queue: list[tuple[str, dict]], db: Database) -> int:   # the data writer, run every few seconds
    n = len(queue)
    for key, value in queue:
        db.rides[key] = value
    queue.clear()
    return n


if __name__ == "__main__":
    db, units, queue = Database(), [], []
    u1, u2 = ProcessingUnit("unit-1", units, queue), ProcessingUnit("unit-2", units, queue)
    print("unit-1 unlocked", u1.unlock("Mia", "B7"), "- grid updated, database write queued")
    print("unit-2 unlocked", u2.unlock("Leo", "B9"))
    print("unit-2 already sees R1:", u2.grid["R1"]["rider"], "| rides in the database:", len(db.rides))
    print("data writer flushed", flush(queue, db), "writes | rides in the database:", len(db.rides))
    units.remove(u2)                              # unit-2 crashes during the surge
    u3 = ProcessingUnit("unit-3", units, queue)
    print("unit-3 started and copied the grid:", sorted(u3.grid), "| units running:", [u.name for u in units])

interface Ride { rider: string; bike: string; open: boolean; }
class Database { rides = new Map<string, Ride>(); } // slow, durable, updated later

// A copy of the application holding its working data in memory, kept in step with its peers.
class ProcessingUnit {
  grid = new Map<string, Ride>();
  constructor(readonly name: string, private peers: ProcessingUnit[], private queue: [string, Ride][]) {
    if (peers.length) for (const [k, v] of peers[0].grid) this.grid.set(k, { ...v }); // a new unit copies the grid from a running peer
    peers.push(this);
  }
  put(key: string, value: Ride): void {
    for (const p of this.peers) p.grid.set(key, { ...value }); // replicate to every unit's memory
    this.queue.push([key, { ...value }]);                     // the database hears about it later
  }
  unlock(rider: string, bike: string): string {
    const id = `R${this.grid.size + 1}`;
    this.put(id, { rider, bike, open: true });
    return id;
  }
}

function flush(queue: [string, Ride][], db: Database): number { // the data writer, run every few seconds
  const n = queue.length;
  for (const [k, v] of queue) db.rides.set(k, v);
  queue.length = 0;
  return n;
}

const db = new Database(), units: ProcessingUnit[] = [], queue: [string, Ride][] = [];
const u1 = new ProcessingUnit("unit-1", units, queue), u2 = new ProcessingUnit("unit-2", units, queue);
console.log(`unit-1 unlocked ${u1.unlock("Mia", "B7")} - grid updated, database write queued`);
console.log(`unit-2 unlocked ${u2.unlock("Leo", "B9")}`);
console.log(`unit-2 already sees R1: ${u2.grid.get("R1")?.rider} | rides in the database: ${db.rides.size}`);
console.log(`data writer flushed ${flush(queue, db)} writes | rides in the database: ${db.rides.size}`);
units.splice(units.indexOf(u2), 1); // unit-2 crashes during the surge
const u3 = new ProcessingUnit("unit-3", units, queue);
console.log(`unit-3 started and copied the grid: [${[...u3.grid.keys()].sort().map((k) => `'${k}'`).join(", ")}] | units running: [${units.map((u) => `'${u.name}'`).join(", ")}]`);

import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.TreeSet;
import java.util.stream.Collectors;

record Ride(String rider, String bike, boolean open) {}

class Database { final Map<String, Ride> rides = new LinkedHashMap<>(); } // slow, durable, updated later

// A copy of the application holding its working data in memory, kept in step with its peers.
class ProcessingUnit {
    final String name; final Map<String, Ride> grid = new LinkedHashMap<>();
    private final List<ProcessingUnit> peers; private final List<Map.Entry<String, Ride>> queue;
    ProcessingUnit(String name, List<ProcessingUnit> peers, List<Map.Entry<String, Ride>> queue) {
        this.name = name; this.peers = peers; this.queue = queue;
        if (!peers.isEmpty()) grid.putAll(peers.get(0).grid); // a new unit copies the grid from a running peer
        peers.add(this);
    }
    void put(String key, Ride value) {
        for (ProcessingUnit p : peers) p.grid.put(key, value); // replicate to every unit's memory
        queue.add(Map.entry(key, value));                     // the database hears about it later
    }
    String unlock(String rider, String bike) {
        String id = "R" + (grid.size() + 1);
        put(id, new Ride(rider, bike, true));
        return id;
    }
}

public class Main {
    static int flush(List<Map.Entry<String, Ride>> queue, Database db) { // the data writer, run every few seconds
        int n = queue.size();
        for (Map.Entry<String, Ride> e : queue) db.rides.put(e.getKey(), e.getValue());
        queue.clear();
        return n;
    }
    public static void main(String[] args) {
        Database db = new Database(); List<ProcessingUnit> units = new ArrayList<>(); List<Map.Entry<String, Ride>> queue = new ArrayList<>();
        ProcessingUnit u1 = new ProcessingUnit("unit-1", units, queue), u2 = new ProcessingUnit("unit-2", units, queue);
        System.out.println("unit-1 unlocked " + u1.unlock("Mia", "B7") + " - grid updated, database write queued");
        System.out.println("unit-2 unlocked " + u2.unlock("Leo", "B9"));
        System.out.println("unit-2 already sees R1: " + u2.grid.get("R1").rider() + " | rides in the database: " + db.rides.size());
        System.out.println("data writer flushed " + flush(queue, db) + " writes | rides in the database: " + db.rides.size());
        units.remove(u2); // unit-2 crashes during the surge
        ProcessingUnit u3 = new ProcessingUnit("unit-3", units, queue);
        System.out.println("unit-3 started and copied the grid: [" + new TreeSet<>(u3.grid.keySet()).stream().map(k -> "'" + k + "'").collect(Collectors.joining(", "))
            + "] | units running: [" + units.stream().map(u -> "'" + u.name + "'").collect(Collectors.joining(", ")) + "]");
    }
}

Output (the same in all three languages)

unit-1 unlocked R1 - grid updated, database write queued
unit-2 unlocked R2
unit-2 already sees R1: Mia | rides in the database: 0
data writer flushed 2 writes | rides in the database: 2
unit-3 started and copied the grid: ['R1', 'R2'] | units running: ['unit-1', 'unit-3']

6. Try it

The write-behind window

During the surge, rides change fast. Choose how often the data writer flushes, how many copies of each item the grid keeps, and how many units fail at once.

Simplified: the database receives one batch per flush; data still waiting for a flush is lost only if every in-memory copy of it is lost.

7. What it costs, and when to pick another style

8. Signs you've outgrown it

• The working set no longer fits in the grid's memory.
• Conflicting updates between units are a regular source of bugs.
• Every write needs to be durable before the user gets an answer.

9. Remember it and say it

In an interview or design review

• Say why the database is off the path: "20,000 unlocks in half an hour; the grid serves them, the database gets batched writes every 5 seconds."
• State the loss window and the copies: "Two copies of each item in memory; we'd lose up to 5 seconds of changes only if both copies fail at once."
• Name products that implement it (Hazelcast, Apache Ignite, GigaSpaces, Coherence) and the simpler alternative (a cache in front of a database).

10. Check yourself

11. Go deeper