InterviewKata
Architecture styles · 12 of 16 · Moving data and messages

Broker

In the broker style, nobody calls a server directly: clients name the service they want, and a broker finds a server that offers it. This page has Pedal's pricing and dock servers register with a broker, hides a server failure from the rider app, and shows in a lab why a registry without heartbeats keeps sending requests to the dead.

See the patternThe class-level versions of a go-between: 19 Mediator · 09 Proxy.

1. In one picture

A switchboard operator: you ask to be put through to "accounts", not to a particular desk. The operator knows which desks handle accounts today, which are busy, and which are empty, and connects you to one. Desks can move, close or be added without anyone changing the number they call.

In the broker style, clients and servers never address each other directly. Servers register the services they offer with a broker; clients send requests to the broker naming the service they want; the broker finds a suitable server, forwards the request, and returns the answer. Proxies on each side make a remote call look like a local method call.

Classic examples are CORBA and Java RMI registries; today the same job is done by service discovery plus a load balancer, API gateways, service meshes, and the request-reply side of message brokers. It differs from an event broker: here a caller wants an answer from exactly one server.

2. Pedal built this way

Two pricing servers and one dock-status server register with the broker. The rider app asks for "quote" and never learns which server answered. Follow what happens when one pricing server dies.

3. The rules

RuleWhy it existsBroken when…
Clients address services, never serversServers can move, scale and fail without clients changing.The app has pricing-1's address in its configuration.
Servers register what they offer, and deregister or expireThe registry reflects reality.A crashed server stays in the registry and gets traffic for an hour.
The broker routes; it doesn't decide business rulesIt stays simple, fast and replaceable.Discount logic added to the broker's routing table.
The broker itself is redundantIt's on every request's path.One broker process; when it restarts, every client fails.

4. How it goes wrong

A single brokerEverything depends on it; run several copies and keep the registry replicated.
Stale registrationsNo heartbeats or expiry, so dead servers keep receiving requests.
Hidden latencyEvery call takes an extra hop through the broker; for hot paths, cache the lookup in the client (client-side discovery).
Logic creeping into the brokerTransformations and rules in the middle become a bottleneck owned by one team: SOA's bus problem again.

5. The code

A broker with a registry of services, two pricing servers and a dock-status server, and a client proxy the rider app calls like a local object. When pricing-1 goes down, the broker retries on pricing-2; an unregistered service gets a clear answer.

from __future__ import annotations
from typing import Callable


class Server:
    def __init__(self, name: str, handlers: dict[str, Callable[[dict], str]]) -> None:
        self.name, self.handlers, self.up = name, handlers, True

    def handle(self, service: str, request: dict) -> str:
        if not self.up:
            raise ConnectionError(self.name)
        return self.handlers[service](request)


class Broker:
    """Clients name a service; the broker finds a server for it, forwards, and tries another if one fails."""

    def __init__(self) -> None:
        self.registry: dict[str, list[Server]] = {}
        self.turn: dict[str, int] = {}

    def register(self, server: Server) -> None:
        for service in server.handlers:
            self.registry.setdefault(service, []).append(server)
        print(f"{server.name} registered {', '.join(server.handlers)}")

    def request(self, service: str, request: dict) -> str:
        servers = self.registry.get(service, [])
        if not servers:
            return f"no server registered for '{service}'"
        tried = []
        for _ in range(len(servers)):
            i = self.turn.get(service, 0)
            self.turn[service] = (i + 1) % len(servers)          # round robin
            server = servers[i]
            try:
                answer = server.handle(service, request)
                return f"{server.name}{' (after ' + ', '.join(tried) + ' failed)' if tried else ''}: {answer}"
            except ConnectionError:
                tried.append(server.name)
        return f"all servers for '{service}' failed"


class ClientProxy:                                   # what the rider app's code calls: looks like a local object
    def __init__(self, broker: Broker) -> None:
        self.broker = broker

    def quote(self, plan: str, minutes: int) -> str:
        return self.broker.request("quote", {"plan": plan, "minutes": minutes})

    def dock_status(self, dock: str) -> str:
        return self.broker.request("dock-status", {"dock": dock})


def price(r: dict) -> str:
    return str(max(0, r["minutes"] - 30) * 10 if r["plan"] == "member" else 100 + r["minutes"] * 10)


if __name__ == "__main__":
    broker = Broker()
    p1, p2 = Server("pricing-1", {"quote": price}), Server("pricing-2", {"quote": price})
    for s in (p1, p2, Server("docks-1", {"dock-status": lambda r: f"{r['dock']} has 1 bike"})):
        broker.register(s)
    app = ClientProxy(broker)
    print("quote(member, 42) ->", app.quote("member", 42))
    print("quote(payg, 15) ->", app.quote("payg", 15))
    p1.up = False
    print("quote(payg, 20) ->", app.quote("payg", 20))
    print("dock_status(D12) ->", app.dock_status("D12"))
    print("weather ->", broker.request("weather", {}))
type Req = { plan?: string; minutes?: number; dock?: string };
type Handler = (r: Req) => string;

class ServerDown extends Error {}

class Server {
  up = true;
  constructor(readonly name: string, readonly handlers: Record<string, Handler>) {}
  handle(service: string, r: Req): string { if (!this.up) throw new ServerDown(this.name); return this.handlers[service](r); }
}

// Clients name a service; the broker finds a server for it, forwards, and tries another if one fails.
class Broker {
  private registry = new Map<string, Server[]>();
  private turn = new Map<string, number>();
  register(s: Server): void {
    for (const service of Object.keys(s.handlers)) this.registry.set(service, [...(this.registry.get(service) ?? []), s]);
    console.log(`${s.name} registered ${Object.keys(s.handlers).join(", ")}`);
  }
  request(service: string, r: Req): string {
    const servers = this.registry.get(service) ?? [];
    if (!servers.length) return `no server registered for '${service}'`;
    const tried: string[] = [];
    for (let k = 0; k < servers.length; k++) {
      const i = this.turn.get(service) ?? 0;
      this.turn.set(service, (i + 1) % servers.length); // round robin
      const s = servers[i];
      try {
        const answer = s.handle(service, r);
        return `${s.name}${tried.length ? " (after " + tried.join(", ") + " failed)" : ""}: ${answer}`;
      } catch (e) { if (!(e instanceof ServerDown)) throw e; tried.push(s.name); }
    }
    return `all servers for '${service}' failed`;
  }
}

class ClientProxy { // what the rider app's code calls: looks like a local object
  constructor(private broker: Broker) {}
  quote(plan: string, minutes: number): string { return this.broker.request("quote", { plan, minutes }); }
  dockStatus(dock: string): string { return this.broker.request("dock-status", { dock }); }
}

const price: Handler = (r) => String(r.plan === "member" ? Math.max(0, (r.minutes ?? 0) - 30) * 10 : 100 + (r.minutes ?? 0) * 10);

const broker = new Broker();
const p1 = new Server("pricing-1", { quote: price }), p2 = new Server("pricing-2", { quote: price });
for (const s of [p1, p2, new Server("docks-1", { "dock-status": (r) => `${r.dock} has 1 bike` })]) broker.register(s);
const app = new ClientProxy(broker);
console.log("quote(member, 42) -> " + app.quote("member", 42));
console.log("quote(payg, 15) -> " + app.quote("payg", 15));
p1.up = false;
console.log("quote(payg, 20) -> " + app.quote("payg", 20));
console.log("dock_status(D12) -> " + app.dockStatus("D12"));
console.log("weather -> " + broker.request("weather", {}));
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;

class ServerDown extends RuntimeException {
    private static final long serialVersionUID = 1L;
    ServerDown(String name) { super(name); }
}

class Server {
    final String name; final Map<String, Function<Map<String, String>, String>> handlers; boolean up = true;
    Server(String name, Map<String, Function<Map<String, String>, String>> handlers) { this.name = name; this.handlers = handlers; }
    String handle(String service, Map<String, String> r) { if (!up) throw new ServerDown(name); return handlers.get(service).apply(r); }
}

// Clients name a service; the broker finds a server for it, forwards, and tries another if one fails.
class Broker {
    private final Map<String, List<Server>> registry = new HashMap<>();
    private final Map<String, Integer> turn = new HashMap<>();
    void register(Server s) {
        for (String service : s.handlers.keySet()) registry.computeIfAbsent(service, k -> new ArrayList<>()).add(s);
        System.out.println(s.name + " registered " + String.join(", ", s.handlers.keySet()));
    }
    String request(String service, Map<String, String> r) {
        List<Server> servers = registry.getOrDefault(service, List.of());
        if (servers.isEmpty()) return "no server registered for '" + service + "'";
        List<String> tried = new ArrayList<>();
        for (int k = 0; k < servers.size(); k++) {
            int i = turn.getOrDefault(service, 0);
            turn.put(service, (i + 1) % servers.size()); // round robin
            Server s = servers.get(i);
            try {
                String answer = s.handle(service, r);
                return s.name + (tried.isEmpty() ? "" : " (after " + String.join(", ", tried) + " failed)") + ": " + answer;
            } catch (ServerDown e) { tried.add(s.name); }
        }
        return "all servers for '" + service + "' failed";
    }
}

class ClientProxy { // what the rider app's code calls: looks like a local object
    private final Broker broker;
    ClientProxy(Broker broker) { this.broker = broker; }
    String quote(String plan, int minutes) { return broker.request("quote", Map.of("plan", plan, "minutes", String.valueOf(minutes))); }
    String dockStatus(String dock) { return broker.request("dock-status", Map.of("dock", dock)); }
}

public class Main {
    static String price(Map<String, String> r) {
        int m = Integer.parseInt(r.get("minutes"));
        return String.valueOf(r.get("plan").equals("member") ? Math.max(0, m - 30) * 10 : 100 + m * 10);
    }
    public static void main(String[] args) {
        Broker broker = new Broker();
        Map<String, Function<Map<String, String>, String>> pricing = new LinkedHashMap<>();
        pricing.put("quote", Main::price);
        Server p1 = new Server("pricing-1", pricing), p2 = new Server("pricing-2", pricing);
        Map<String, Function<Map<String, String>, String>> docks = new LinkedHashMap<>();
        docks.put("dock-status", r -> r.get("dock") + " has 1 bike");
        for (Server s : List.of(p1, p2, new Server("docks-1", docks))) broker.register(s);
        ClientProxy app = new ClientProxy(broker);
        System.out.println("quote(member, 42) -> " + app.quote("member", 42));
        System.out.println("quote(payg, 15) -> " + app.quote("payg", 15));
        p1.up = false;
        System.out.println("quote(payg, 20) -> " + app.quote("payg", 20));
        System.out.println("dock_status(D12) -> " + app.dockStatus("D12"));
        System.out.println("weather -> " + broker.request("weather", Map.of()));
    }
}

Output (the same in all three languages)

pricing-1 registered quote
pricing-2 registered quote
docks-1 registered dock-status
quote(member, 42) -> pricing-1: 120
quote(payg, 15) -> pricing-2: 250
quote(payg, 20) -> pricing-2 (after pricing-1 failed): 300
dock_status(D12) -> docks-1: D12 has 1 bike
weather -> no server registered for 'weather'

6. Try it

Route twelve requests

Three pricing servers are registered. Take some down, then send twelve quote requests through the broker (round robin, retrying on failure).

Registry

7. What it costs, and when to pick another style

What you get

  • Location transparency: clients don't know where or how many servers there are.
  • Easy scaling and failover: add servers by registering them; the broker routes around failures.
  • Heterogeneous servers: different languages and machines behind one way of calling.
  • Central place for cross-cutting work: authentication, metrics, rate limits.

What you pay

  • An extra hop: latency on every request.
  • A critical dependency: the broker must be highly available.
  • Harder debugging: the path of a request goes through a middleman.
  • Temptation to centralise logic in the broker.
If…Consider instead
Fire-and-forget notifications to many listenersEvent-driven
A few fixed servers that rarely changeDirect calls in client-server
Many services owned by many teamsMicroservices with service discovery or a mesh
No central party at allPeer-to-peer

8. Signs you've outgrown it

9. Remember it and say it

Memory hook: "The switchboard"Clients ask for a service by name; a broker finds a server that offers it and connects them.
How this hook works: what “The switchboard” means

What the slogan means, phrase by phrase

“The switchboard”
A broker between all callers and servers.
“by name”
Clients name the service, never the machine.

Why this picture fits: A switchboard operator puts you through to "accounts", whichever desk is free today.

The story → the concept

The switchboardPedal
CallersThe rider app's client proxy
The operatorThe broker
The directory of desksThe registry: quote → pricing-1, pricing-2
An empty deskpricing-1 down: put through to pricing-2

Watch it on an example

  1. The app asks for "quote"; it never knows which server answers.
  2. pricing-1 dies; the broker retries on pricing-2.
  3. An unknown service gets a clear answer, not a hang.

The trap it saves you from

Running one broker, so the thing that hides failures becomes the biggest failure.

It still works when…

Under other names: API gateways, service discovery and service meshes all play the broker's part.

In an interview or design review

10. Check yourself

What does a client in the broker style address?

Location transparency: the broker resolves names to servers.

A crashed server keeps receiving requests for an hour. What's missing?

Registrations must reflect which servers are actually alive.

How does a broker differ from an event broker?

Request-reply vs publish-subscribe.

What's the main operational risk of a broker?

Run it redundantly.

11. Go deeper

ResourceWhy
System design D38: API gatewayA modern broker in front of hundreds of services.
System design 08: Load balancingHealth checks and routing policies.
Consul service discoveryRegistration, health checks and lookup in practice.