Shopify: the modular monolith

Shopify's core is one of the largest Ruby on Rails applications in existence: over 2.8 million lines of Ruby, worked on by more than a thousand developers. When the monolith's downsides started to outweigh its benefits, Shopify did not split it into microservices. It kept one codebase and one deployment, reorganised the code by business domain, and built tooling that rejects a pull request when one component reaches into another's internals. Only two pieces were split out, each for a reason that can be stated in one sentence.

1. The situation

Shopify grew for over a decade as a single Rails monolith: billing merchants, third-party apps, products, checkout, shipping, all in one codebase with no boundaries between them. The code that calculated shipping rates lived alongside checkout code, "and there was very little stopping them from calling each other". The result was extremely high coupling: changes in one place broke things elsewhere, and developers needed context on the whole system to change any part of it.

Before choosing a remedy, the team surveyed every developer working in the core system about their main pain points, "to be data-informed", and in early 2017 a small team started the work.

2. What mattered

• Developer productivity: new developers should be able to learn only the part they work on; tests should run on the components a change affects.
• Modifiability: change one component freely as long as its contracts hold.
• Keep what the monolith does well: one deployment, one test suite, no network calls between parts of the business.

3. The options

4. The decision

Decision: Evolve the core into a modular monolith ("Componentization")

Context: One Rails codebase, 10+ years, 1,000+ developers, no internal
  boundaries; a developer survey named coupling and lack of ownership as the
  main pain points.
Decision: Keep one codebase and one deployment. Reorganise code by business
  domain (orders, shipping, inventory, billing) instead of by Rails concept
  (models, views, controllers). Each component exposes a public API and
  exclusively owns its data. Enforce boundaries with tooling in CI.
  Extract a separate service only for a specific reason.
Consequences:
  + Developers work in what feels like a smaller app
  + No network between business domains; one deploy, one test suite
  - Boundaries need tooling and a cultural shift; progress is gradual
  - A dense, cyclic dependency graph must be untangled component by component
Compliance: dependency and privacy violations rejected in pull requests

5. How it works

From folders to enforced boundaries, in the order Shopify did it.

6. The numbers

• About 6,000 Ruby classes listed in a spreadsheet and labelled, by hand, with the component each belongs to, then moved in one big pull request built by scripts (only file moves, so tests caught any mistakes; one side effect was that GitHub lost the files' history).
• The first boundary tool, Wedge, traced every call during the test suite and took over an hour; its output was hard to act on.
• Its replacement, Packwerk, analyses static constant references, runs on the largest codebase in a few minutes, and runs on every pull request.
• By September 2020: 37 components with public entry points; Packwerk enforced on about a third of them. Long-term goal: an acyclic dependency graph.

7. What happened

Eighteen months after the first post, Shopify reported that it stood by the original ideas but that "almost all of the details have changed". The biggest lesson was that public interfaces alone didn't help: when the dependency graph was ignored, a public interface "turned out to just be an added layer of indirection", spreading a design problem into more files. The work shifted to the dependency graph: removing cycles and reducing how much depends on central entities like Shop. Discussions about which way a dependency should point regularly exposed objects with too many responsibilities and missing abstractions. Packwerk was later released as open source.

Shopify kept splitting rare and deliberate. Two examples from the 2020 post: storefront rendering became a separate service because it is a read-only, very high-throughput use case that makes sense to scale and distribute apart from the merchant admin; and credit card vaulting is separate because it processes sensitive data that shouldn't flow through the rest of the system.

8. Lessons

• Modularity and distribution are separate decisions. Domain boundaries fix coupling; services add independent deployment and scaling, at a price. Shopify bought the first everywhere and the second only twice (Module 11, 10).
• Organise by domain, not by technical layer. Moving from models/views/controllers to orders/shipping/billing is domain partitioning (Module 08, 08a).
• A boundary that isn't checked isn't a boundary. Packwerk in every pull request is a structural fitness function (Module 26). Fast and actionable beat complete: Wedge's hour-long trace lost to a few minutes of static analysis.
• Watch the dependency graph, not just the interfaces. Cycles and a few central entities keep a modular monolith coupled (Module 07).
• Every extracted service has a one-sentence reason. Different scaling (storefront) or different security (card vault): the characteristics differ, so the quantum differs (Module 09).

9. Your turn

1. Label the classes (15 min). Take 30 files from a codebase you know and label each with a business component. Which files don't fit anywhere?
2. Draw the graph (10 min). Draw dependencies between those components. Where are the cycles, and what does everything depend on?
3. Justify one split (5 min). Name one part you would extract as a service, in one sentence about a characteristic that differs.

10. Check yourself

11. Sources and further reading