SoundCloud: strangling the mothership

SoundCloud's product ran on one Ruby on Rails application its engineers called "the mothership". When the team measured it, shipping a feature took about two months, most of it spent waiting. The company didn't rewrite the monolith. It built every new feature outside it, gave teams ownership of the old parts so they'd actually be extracted, and put "stranglers" in front until the monolith no longer faced the internet. The engineer who led the move says the reason was productivity, not technology.

1. The situation

Two teams built SoundCloud. The App team owned the Rails monolith and the public API inside it. The Web team built "The Next SoundCloud", a complete revamp of the website as a single-page JavaScript application that was "a regular client to our public API". The teams "were really isolated—we even lived in separate buildings across Berlin."

A value stream map showed where the time went: "a feature would take about two months to go live", and "out of the 47 days spent in engineering, only 11 days are doing actual work." The rest was waiting: in a shared document, on a Trello board, in a Pivotal Tracker queue, and in a deploy queue.

47 days in engineering, from the post's value stream map.

2. What mattered

• Lead time. Two months per feature, with three quarters of the engineering time spent waiting.
• Handoffs. Every feature crossed two teams in two buildings.
• Review queues. Large pull requests against a monolith nobody fully understood waited a long time for review.
• Keep the product running while any of this changed.

3. What they tried

4. The decision

Decision: Strangle the monolith instead of rewriting it

Context: A Rails monolith ("the mothership") with the public API inside;
  ~2 months per feature, 11 of 47 engineering days doing actual work;
  extraction of old features kept being postponed.
Decision:
  - Build every new feature as its own service, outside the monolith.
  - Give small teams ownership of existing modules, so extraction has an
    owner instead of being "everybody's responsibility".
  - Put stranglers in front of the monolith, so traffic moves feature by
    feature and the monolith stops facing the internet.
Consequences:
  + New work ships without the monolith's queues
  + Old code leaves because someone owns it
  - Every service adds a fixed cost to run
  - Two systems to operate until the monolith is off the critical path

5. How it works

From one monolith with a public API to a monolith hidden behind stranglers.

6. The technique in code

An interceptor in front of the monolith. For the feature being extracted, it copies writes to the new service and compares reads; a backfill fixes the old records the copy missed; then the feature switches to the service. The program was run in all three languages; each prints the output below it.

class Store:
    """The monolith, or a new service: a dict of records with read and write."""

    def __init__(self, name, records=None):
        self.name, self.records = name, dict(records or {})

    def handle(self, op, key, value=None):
        if op == "write":
            self.records[key] = value
        return self.records.get(key, "missing")


class Interceptor:
    """Sits in front of the monolith; per feature: monolith, copy (writes copied, reads compared) or service."""

    def __init__(self, monolith):
        self.monolith, self.services, self.mode, self.mismatches = monolith, {}, {}, []

    def extract(self, feature, service):
        self.services[feature], self.mode[feature] = service, "copy"

    def handle(self, feature, op, key, value=None):
        mode = self.mode.get(feature, "monolith")
        if mode == "service":
            return self.services[feature].handle(op, key, value)
        answer = self.monolith.handle(op, key, value)
        if mode == "copy":
            shadow = self.services[feature].handle(op, key, value)
            if op == "read" and shadow != answer:
                self.mismatches.append(f"{feature}/{key}: monolith {answer}, service {shadow}")
        return answer

    def on_monolith(self, features):
        return [f for f in features if self.mode.get(f, "monolith") != "service"]


FEATURES = ["tracks", "messages", "stats"]
mothership = Store("monolith", {"m1": "hi", "m2": "new track out"})
edge = Interceptor(mothership)

edge.extract("messages", Store("messages service"))
edge.handle("messages", "write", "m3", "great set")          # copied to both
for key in ["m1", "m2", "m3"]:
    edge.handle("messages", "read", key)
print("copy mode:", len(edge.mismatches), "mismatches")
for m in edge.mismatches:
    print("  " + m)

for key, value in mothership.records.items():                 # backfill the old messages
    edge.services["messages"].handle("write", key, value)
edge.mismatches.clear()
for key in ["m1", "m2", "m3"]:
    edge.handle("messages", "read", key)
print("after backfill:", len(edge.mismatches), "mismatches")

edge.mode["messages"] = "service"
edge.handle("messages", "write", "m4", "see you in Berlin")
print("service mode: m4 in service:", edge.services["messages"].handle("read", "m4"), "| in monolith:", mothership.handle("read", "m4"))
print("still on the monolith:", ", ".join(edge.on_monolith(FEATURES)))

class Store {
  // The monolith, or a new service: a map of records with read and write.
  records: Map<string, string>;
  constructor(public name: string, records: [string, string][] = []) { this.records = new Map(records); }

  handle(op: "read" | "write", key: string, value?: string): string {
    if (op === "write") this.records.set(key, value!);
    return this.records.get(key) ?? "missing";
  }
}

type Mode = "monolith" | "copy" | "service";

class Interceptor {
  // Sits in front of the monolith; per feature: monolith, copy (writes copied, reads compared) or service.
  services = new Map<string, Store>();
  mode = new Map<string, Mode>();
  mismatches: string[] = [];
  constructor(private monolith: Store) {}

  extract(feature: string, service: Store) { this.services.set(feature, service); this.mode.set(feature, "copy"); }

  handle(feature: string, op: "read" | "write", key: string, value?: string): string {
    const mode = this.mode.get(feature) ?? "monolith";
    if (mode === "service") return this.services.get(feature)!.handle(op, key, value);
    const answer = this.monolith.handle(op, key, value);
    if (mode === "copy") {
      const shadow = this.services.get(feature)!.handle(op, key, value);
      if (op === "read" && shadow !== answer) this.mismatches.push(`${feature}/${key}: monolith ${answer}, service ${shadow}`);
    }
    return answer;
  }

  onMonolith(features: string[]) { return features.filter((f) => (this.mode.get(f) ?? "monolith") !== "service"); }
}

const FEATURES = ["tracks", "messages", "stats"];
const mothership = new Store("monolith", [["m1", "hi"], ["m2", "new track out"]]);
const edge = new Interceptor(mothership);

edge.extract("messages", new Store("messages service"));
edge.handle("messages", "write", "m3", "great set");          // copied to both
for (const key of ["m1", "m2", "m3"]) edge.handle("messages", "read", key);
console.log(`copy mode: ${edge.mismatches.length} mismatches`);
for (const m of edge.mismatches) console.log("  " + m);

for (const [key, value] of mothership.records) edge.services.get("messages")!.handle("write", key, value);   // backfill the old messages
edge.mismatches = [];
for (const key of ["m1", "m2", "m3"]) edge.handle("messages", "read", key);
console.log(`after backfill: ${edge.mismatches.length} mismatches`);

edge.mode.set("messages", "service");
edge.handle("messages", "write", "m4", "see you in Berlin");
console.log(`service mode: m4 in service: ${edge.services.get("messages")!.handle("read", "m4")} | in monolith: ${mothership.handle("read", "m4")}`);
console.log(`still on the monolith: ${edge.onMonolith(FEATURES).join(", ")}`);

import java.util.*;

public class Main {
    static class Store {
        // The monolith, or a new service: a map of records with read and write.
        final String name;
        final Map<String, String> records = new LinkedHashMap<>();
        Store(String name) { this.name = name; }

        String handle(String op, String key, String value) {
            if (op.equals("write")) records.put(key, value);
            return records.getOrDefault(key, "missing");
        }
    }

    static class Interceptor {
        // Sits in front of the monolith; per feature: monolith, copy (writes copied, reads compared) or service.
        final Store monolith;
        final Map<String, Store> services = new HashMap<>();
        final Map<String, String> mode = new HashMap<>();
        final List<String> mismatches = new ArrayList<>();
        Interceptor(Store monolith) { this.monolith = monolith; }

        void extract(String feature, Store service) { services.put(feature, service); mode.put(feature, "copy"); }

        String handle(String feature, String op, String key, String value) {
            String m = mode.getOrDefault(feature, "monolith");
            if (m.equals("service")) return services.get(feature).handle(op, key, value);
            String answer = monolith.handle(op, key, value);
            if (m.equals("copy")) {
                String shadow = services.get(feature).handle(op, key, value);
                if (op.equals("read") && !shadow.equals(answer)) mismatches.add(feature + "/" + key + ": monolith " + answer + ", service " + shadow);
            }
            return answer;
        }

        List<String> onMonolith(List<String> features) {
            List<String> out = new ArrayList<>();
            for (String f : features) if (!mode.getOrDefault(f, "monolith").equals("service")) out.add(f);
            return out;
        }
    }

    public static void main(String[] args) {
        List<String> features = List.of("tracks", "messages", "stats");
        Store mothership = new Store("monolith");
        mothership.records.put("m1", "hi");
        mothership.records.put("m2", "new track out");
        Interceptor edge = new Interceptor(mothership);

        edge.extract("messages", new Store("messages service"));
        edge.handle("messages", "write", "m3", "great set");          // copied to both
        for (String key : List.of("m1", "m2", "m3")) edge.handle("messages", "read", key, null);
        System.out.println("copy mode: " + edge.mismatches.size() + " mismatches");
        for (String m : edge.mismatches) System.out.println("  " + m);

        for (Map.Entry<String, String> e : mothership.records.entrySet()) edge.services.get("messages").handle("write", e.getKey(), e.getValue());   // backfill the old messages
        edge.mismatches.clear();
        for (String key : List.of("m1", "m2", "m3")) edge.handle("messages", "read", key, null);
        System.out.println("after backfill: " + edge.mismatches.size() + " mismatches");

        edge.mode.put("messages", "service");
        edge.handle("messages", "write", "m4", "see you in Berlin");
        System.out.println("service mode: m4 in service: " + edge.services.get("messages").handle("read", "m4", null) + " | in monolith: " + mothership.handle("read", "m4", null));
        System.out.println("still on the monolith: " + String.join(", ", edge.onMonolith(features)));
    }
}

copy mode: 2 mismatches
  messages/m1: monolith hi, service missing
  messages/m2: monolith new track out, service missing
after backfill: 0 mismatches
service mode: m4 in service: see you in Berlin | in monolith: missing
still on the monolith: tracks, stats

7. The numbers

• About two months for a feature to go live before the change.
• 11 of 47 days in engineering were actual work.
• 2 teams of 2 engineers delivered the monetisation project, built as services, "ahead of deadlines".
• Not internet-facing: by September 2015, "due to a system of stranglers", the monolith no longer took traffic directly. The author expected it to be off every critical path within a year.

Try it: strangle the mothership

Put stranglers at the edge, give each feature an owner, then extract it. The features are an invented list shaped like SoundCloud's product. A feature without an owner can't move: that's the post's point about "everybody's responsibility".

8. What happened

New features shipped as services, starting with the monetisation project. Teams that owned modules extracted messaging, stats and other parts of the monolith. By the time of the post, the stranglers carried all external traffic, and the monolith was on its way off the critical path. The author is clear about the motive: "the reason we migrated to microservices had to do much more with productivity than pure technical matters." He also warns that "as organisations grow, they need to be mindful about the fixed cost of each service."

9. Lessons

• Measure where the time goes first. A value stream map showed waiting, not coding, was the problem. Architecture changed because the process data pointed there (Module 09a).
• Stop adding to the monolith before you try to shrink it. New features outside it cost nothing to extract later.
• Extraction needs an owner. "What's everybody's responsibility is nobody's responsibility": give each old module a team.
• Strangle at the edge. Once all traffic passes through the stranglers, each feature can move with a routing change (Module 30a).
• Services have a fixed cost. Each one needs deploys, monitoring and on-call. SoundCloud's reason was productivity; check that each split pays (C07 Segment shows the opposite trade).

10. Your turn

1. Map one feature (10 min). For the last feature you shipped, list every queue it waited in, and estimate work days against waiting days.
2. Find the edge (10 min). Where could a strangler sit in front of your oldest system: a gateway, a proxy, the client?
3. Assign the owners (10 min). List the old system's modules. Which have a team that would extract them? Which are "everybody's responsibility"?

11. Check yourself

12. Sources and further reading