Broker
In the broker style, nobody calls a server directly: clients name the service they want, and a broker finds a server that offers it. This page has Pedal's pricing and dock servers register with a broker, hides a server failure from the rider app, and shows in a lab why a registry without heartbeats keeps sending requests to the dead.
1. In one picture
In the broker style, clients and servers never address each other directly. Servers register the services they offer with a broker; clients send requests to the broker naming the service they want; the broker finds a suitable server, forwards the request, and returns the answer. Proxies on each side make a remote call look like a local method call.
Classic examples are CORBA and Java RMI registries; today the same job is done by service discovery plus a load balancer, API gateways, service meshes, and the request-reply side of message brokers. It differs from an event broker: here a caller wants an answer from exactly one server.
2. Pedal built this way
Two pricing servers and one dock-status server register with the broker. The rider app asks for "quote" and never learns which server answered. Follow what happens when one pricing server dies.
3. The rules
| Rule | Why it exists | Broken when… |
|---|---|---|
| Clients address services, never servers | Servers can move, scale and fail without clients changing. | The app has pricing-1's address in its configuration. |
| Servers register what they offer, and deregister or expire | The registry reflects reality. | A crashed server stays in the registry and gets traffic for an hour. |
| The broker routes; it doesn't decide business rules | It stays simple, fast and replaceable. | Discount logic added to the broker's routing table. |
| The broker itself is redundant | It's on every request's path. | One broker process; when it restarts, every client fails. |
4. How it goes wrong
5. The code
A broker with a registry of services, two pricing servers and a dock-status server, and a client proxy the rider app calls like a local object. When pricing-1 goes down, the broker retries on pricing-2; an unregistered service gets a clear answer.
from __future__ import annotations
from typing import Callable
class Server:
def __init__(self, name: str, handlers: dict[str, Callable[[dict], str]]) -> None:
self.name, self.handlers, self.up = name, handlers, True
def handle(self, service: str, request: dict) -> str:
if not self.up:
raise ConnectionError(self.name)
return self.handlers[service](request)
class Broker:
"""Clients name a service; the broker finds a server for it, forwards, and tries another if one fails."""
def __init__(self) -> None:
self.registry: dict[str, list[Server]] = {}
self.turn: dict[str, int] = {}
def register(self, server: Server) -> None:
for service in server.handlers:
self.registry.setdefault(service, []).append(server)
print(f"{server.name} registered {', '.join(server.handlers)}")
def request(self, service: str, request: dict) -> str:
servers = self.registry.get(service, [])
if not servers:
return f"no server registered for '{service}'"
tried = []
for _ in range(len(servers)):
i = self.turn.get(service, 0)
self.turn[service] = (i + 1) % len(servers) # round robin
server = servers[i]
try:
answer = server.handle(service, request)
return f"{server.name}{' (after ' + ', '.join(tried) + ' failed)' if tried else ''}: {answer}"
except ConnectionError:
tried.append(server.name)
return f"all servers for '{service}' failed"
class ClientProxy: # what the rider app's code calls: looks like a local object
def __init__(self, broker: Broker) -> None:
self.broker = broker
def quote(self, plan: str, minutes: int) -> str:
return self.broker.request("quote", {"plan": plan, "minutes": minutes})
def dock_status(self, dock: str) -> str:
return self.broker.request("dock-status", {"dock": dock})
def price(r: dict) -> str:
return str(max(0, r["minutes"] - 30) * 10 if r["plan"] == "member" else 100 + r["minutes"] * 10)
if __name__ == "__main__":
broker = Broker()
p1, p2 = Server("pricing-1", {"quote": price}), Server("pricing-2", {"quote": price})
for s in (p1, p2, Server("docks-1", {"dock-status": lambda r: f"{r['dock']} has 1 bike"})):
broker.register(s)
app = ClientProxy(broker)
print("quote(member, 42) ->", app.quote("member", 42))
print("quote(payg, 15) ->", app.quote("payg", 15))
p1.up = False
print("quote(payg, 20) ->", app.quote("payg", 20))
print("dock_status(D12) ->", app.dock_status("D12"))
print("weather ->", broker.request("weather", {}))
type Req = { plan?: string; minutes?: number; dock?: string };
type Handler = (r: Req) => string;
class ServerDown extends Error {}
class Server {
up = true;
constructor(readonly name: string, readonly handlers: Record<string, Handler>) {}
handle(service: string, r: Req): string { if (!this.up) throw new ServerDown(this.name); return this.handlers[service](r); }
}
// Clients name a service; the broker finds a server for it, forwards, and tries another if one fails.
class Broker {
private registry = new Map<string, Server[]>();
private turn = new Map<string, number>();
register(s: Server): void {
for (const service of Object.keys(s.handlers)) this.registry.set(service, [...(this.registry.get(service) ?? []), s]);
console.log(`${s.name} registered ${Object.keys(s.handlers).join(", ")}`);
}
request(service: string, r: Req): string {
const servers = this.registry.get(service) ?? [];
if (!servers.length) return `no server registered for '${service}'`;
const tried: string[] = [];
for (let k = 0; k < servers.length; k++) {
const i = this.turn.get(service) ?? 0;
this.turn.set(service, (i + 1) % servers.length); // round robin
const s = servers[i];
try {
const answer = s.handle(service, r);
return `${s.name}${tried.length ? " (after " + tried.join(", ") + " failed)" : ""}: ${answer}`;
} catch (e) { if (!(e instanceof ServerDown)) throw e; tried.push(s.name); }
}
return `all servers for '${service}' failed`;
}
}
class ClientProxy { // what the rider app's code calls: looks like a local object
constructor(private broker: Broker) {}
quote(plan: string, minutes: number): string { return this.broker.request("quote", { plan, minutes }); }
dockStatus(dock: string): string { return this.broker.request("dock-status", { dock }); }
}
const price: Handler = (r) => String(r.plan === "member" ? Math.max(0, (r.minutes ?? 0) - 30) * 10 : 100 + (r.minutes ?? 0) * 10);
const broker = new Broker();
const p1 = new Server("pricing-1", { quote: price }), p2 = new Server("pricing-2", { quote: price });
for (const s of [p1, p2, new Server("docks-1", { "dock-status": (r) => `${r.dock} has 1 bike` })]) broker.register(s);
const app = new ClientProxy(broker);
console.log("quote(member, 42) -> " + app.quote("member", 42));
console.log("quote(payg, 15) -> " + app.quote("payg", 15));
p1.up = false;
console.log("quote(payg, 20) -> " + app.quote("payg", 20));
console.log("dock_status(D12) -> " + app.dockStatus("D12"));
console.log("weather -> " + broker.request("weather", {}));import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
class ServerDown extends RuntimeException {
private static final long serialVersionUID = 1L;
ServerDown(String name) { super(name); }
}
class Server {
final String name; final Map<String, Function<Map<String, String>, String>> handlers; boolean up = true;
Server(String name, Map<String, Function<Map<String, String>, String>> handlers) { this.name = name; this.handlers = handlers; }
String handle(String service, Map<String, String> r) { if (!up) throw new ServerDown(name); return handlers.get(service).apply(r); }
}
// Clients name a service; the broker finds a server for it, forwards, and tries another if one fails.
class Broker {
private final Map<String, List<Server>> registry = new HashMap<>();
private final Map<String, Integer> turn = new HashMap<>();
void register(Server s) {
for (String service : s.handlers.keySet()) registry.computeIfAbsent(service, k -> new ArrayList<>()).add(s);
System.out.println(s.name + " registered " + String.join(", ", s.handlers.keySet()));
}
String request(String service, Map<String, String> r) {
List<Server> servers = registry.getOrDefault(service, List.of());
if (servers.isEmpty()) return "no server registered for '" + service + "'";
List<String> tried = new ArrayList<>();
for (int k = 0; k < servers.size(); k++) {
int i = turn.getOrDefault(service, 0);
turn.put(service, (i + 1) % servers.size()); // round robin
Server s = servers.get(i);
try {
String answer = s.handle(service, r);
return s.name + (tried.isEmpty() ? "" : " (after " + String.join(", ", tried) + " failed)") + ": " + answer;
} catch (ServerDown e) { tried.add(s.name); }
}
return "all servers for '" + service + "' failed";
}
}
class ClientProxy { // what the rider app's code calls: looks like a local object
private final Broker broker;
ClientProxy(Broker broker) { this.broker = broker; }
String quote(String plan, int minutes) { return broker.request("quote", Map.of("plan", plan, "minutes", String.valueOf(minutes))); }
String dockStatus(String dock) { return broker.request("dock-status", Map.of("dock", dock)); }
}
public class Main {
static String price(Map<String, String> r) {
int m = Integer.parseInt(r.get("minutes"));
return String.valueOf(r.get("plan").equals("member") ? Math.max(0, m - 30) * 10 : 100 + m * 10);
}
public static void main(String[] args) {
Broker broker = new Broker();
Map<String, Function<Map<String, String>, String>> pricing = new LinkedHashMap<>();
pricing.put("quote", Main::price);
Server p1 = new Server("pricing-1", pricing), p2 = new Server("pricing-2", pricing);
Map<String, Function<Map<String, String>, String>> docks = new LinkedHashMap<>();
docks.put("dock-status", r -> r.get("dock") + " has 1 bike");
for (Server s : List.of(p1, p2, new Server("docks-1", docks))) broker.register(s);
ClientProxy app = new ClientProxy(broker);
System.out.println("quote(member, 42) -> " + app.quote("member", 42));
System.out.println("quote(payg, 15) -> " + app.quote("payg", 15));
p1.up = false;
System.out.println("quote(payg, 20) -> " + app.quote("payg", 20));
System.out.println("dock_status(D12) -> " + app.dockStatus("D12"));
System.out.println("weather -> " + broker.request("weather", Map.of()));
}
}Output (the same in all three languages)
pricing-1 registered quote pricing-2 registered quote docks-1 registered dock-status quote(member, 42) -> pricing-1: 120 quote(payg, 15) -> pricing-2: 250 quote(payg, 20) -> pricing-2 (after pricing-1 failed): 300 dock_status(D12) -> docks-1: D12 has 1 bike weather -> no server registered for 'weather'
6. Try it
Route twelve requests
Three pricing servers are registered. Take some down, then send twelve quote requests through the broker (round robin, retrying on failure).
7. What it costs, and when to pick another style
What you get
- Location transparency: clients don't know where or how many servers there are.
- Easy scaling and failover: add servers by registering them; the broker routes around failures.
- Heterogeneous servers: different languages and machines behind one way of calling.
- Central place for cross-cutting work: authentication, metrics, rate limits.
What you pay
- An extra hop: latency on every request.
- A critical dependency: the broker must be highly available.
- Harder debugging: the path of a request goes through a middleman.
- Temptation to centralise logic in the broker.
| If… | Consider instead |
|---|---|
| Fire-and-forget notifications to many listeners | Event-driven |
| A few fixed servers that rarely change | Direct calls in client-server |
| Many services owned by many teams | Microservices with service discovery or a mesh |
| No central party at all | Peer-to-peer |
8. Signs you've outgrown it
- The broker is the slowest hop on the hot path, even after scaling it.
- More and more business rules live in its configuration.
- Clients start bypassing it with hard-coded addresses to save time.
9. Remember it and say it
How this hook works: what “The switchboard” means
What the slogan means, phrase by phrase
- “The switchboard”
- A broker between all callers and servers.
- “by name”
- Clients name the service, never the machine.
Why this picture fits: A switchboard operator puts you through to "accounts", whichever desk is free today.
The story → the concept
| The switchboard | Pedal |
|---|---|
| Callers | The rider app's client proxy |
| The operator | The broker |
| The directory of desks | The registry: quote → pricing-1, pricing-2 |
| An empty desk | pricing-1 down: put through to pricing-2 |
Watch it on an example
- The app asks for "quote"; it never knows which server answers.
- pricing-1 dies; the broker retries on pricing-2.
- An unknown service gets a clear answer, not a hang.
The trap it saves you from
Running one broker, so the thing that hides failures becomes the biggest failure.
It still works when…
Under other names: API gateways, service discovery and service meshes all play the broker's part.
In an interview or design review
- Say how servers are found: registration with heartbeats, a registry (Consul, etcd, the platform's DNS), and routing by the broker or by the client.
- Make the broker redundant and say how: several copies behind DNS or anycast, replicated registry.
- Compare with direct discovery: server-side (broker routes) vs client-side (client looks up, then calls directly).
10. Check yourself
What does a client in the broker style address?
A crashed server keeps receiving requests for an hour. What's missing?
How does a broker differ from an event broker?
What's the main operational risk of a broker?
11. Go deeper
| Resource | Why |
|---|---|
| System design D38: API gateway | A modern broker in front of hundreds of services. |
| System design 08: Load balancing | Health checks and routing policies. |
| Consul service discovery | Registration, health checks and lookup in practice. |