Software architecture · Level 5: Data and transactions · Module 23 of 46

Data in distributed architectures

Splitting code into services is the easy half. The hard half is the data: who owns each table, how others read it, how reports are produced when the data lives in a dozen databases, and how a schema changes without stopping the business. Most distributed architectures that disappoint do so because of data decisions, not code decisions. This module covers ownership, CQRS, reporting, and migrating data while the system runs.

1. The idea

A shared kitchen versus flatmates each with their own cupboard. In a shared kitchen anyone can rearrange anything, so nobody can rely on where things are. With a cupboard each, you own your stuff and others ask before borrowing. That's calmer, but cooking a meal that needs everyone's ingredients takes coordination, and someone has to keep a list of what's in every cupboard.

Who owns the data?

The rule: every piece of data has exactly one owner that writes it. Others read it through the owner's API or events. When two services both want to write the same table, there are three ways out:

Option	How	Cost
Split the table	Each service owns the columns it writes (inventory owns stock counts, catalogue owns descriptions)	Reassembling the whole needs two reads
Data domain	The services share a small schema they co-own, governed together	Back to shared-schema coupling, on purpose and in a small area
Delegate	One service owns it; the other sends it requests to change it	A dependency, sync or async, on every write

CQRS: separate the writes from the reads

Command Query Responsibility Segregation keeps one model for changing data (normalised, validated, transactional) and one or more models for reading it (denormalised, shaped for each screen or report), kept in step by events. Reads become fast and simple; the price is that reads are eventually consistent with writes.

2. How it works

CQRS: an order written once, read many ways

Four ways to feed a reporting service

Finance needs a daily report combining orders, payments and refunds, each owned by a different service. Watch the four options, and what each one breaks.

Option	Good	Bad
Database pull	Simple, fast to build	Reads services' private tables: couples them to every schema change
HTTP pull	Respects ownership	Slow for bulk data; loads the services; fails if any is down
Batch pull	Handles volume	Data a day old; still coupled to schemas if it reads tables
Event push	Decoupled, near real time, owners stay private	Eventual consistency; events must carry what reports need

3. Try it yourself

Migrating 10 million customer records

The customer address format is changing. Choose a migration strategy and see the downtime and how quickly records are converted. With lazy migration, a record is converted the first time it's read; popular records (20% of customers get 80% of reads) convert first.

Assumes 2 million reads a day spread 80/20 over the records, a big-bang conversion rate of 2,000 records a second with the system stopped, and a background job converting 20 records a second during quiet hours (about 1.7 million a day). A record converted on first read costs that read a few extra milliseconds.

4. Trade-offs

Ownership vs convenience

One owner per table keeps services independent; a shared table is convenient until either side needs to change it. Pay the cost of an API or an event up front.

CQRS speed vs consistency

Read models shaped for each screen are fast and simple to query, and lag the writes by milliseconds to seconds. Show "updating" where users would notice.

Fresh reports vs decoupled reports

Reading live tables gives fresh numbers and couples reporting to every schema. Event-fed reporting stores are decoupled and slightly behind; most reports can live with that.

Downtime vs complexity

A big-bang migration is simple and needs a maintenance window. Lazy migration needs no downtime, and code that handles both formats until the last record converts.

5. In real projects

Shared databases change through a process. When several services still share a database, schema changes go through owners of every affected service, using expand and contract: add the new column, write both, move readers, then remove the old one.

Data services (a service whose job is to own and serve one kind of data) can help, if they own business rules too. A pure CRUD data service is the entity trap of Module 08 at the network level.

Reporting usually ends up event-fed. A data warehouse or lake receives change events (or change data capture from each service's database) and builds report-shaped tables. Owners stay private; analysts get everything in one place. Module 23a covers how that analytical side is stored and who owns it.

Say it in the interview

Two-sentence answers

"Two services need to write the same table?" Split the table so each owns the columns it writes, or make one the owner and have the other ask it to make changes. A small co-owned data domain is the last resort.
"What is CQRS?" Separate models for writes and reads, with read models updated from events. Reads get fast and simple; they're eventually consistent with writes.
"How do you do reporting across microservices?" Services publish events (or we capture changes from their databases) into a reporting store. Reporting never reads services' private tables.

Red flags interviewers notice

The reporting team reading every service's production database.
Two services writing the same table, with no owner.
A schema migration plan that's "a weekend of downtime" for a system that can't stop.

6. Remember it

Memory hook: "One writer, many readers, never through the back door"Every table has one owner that writes it. Everyone else reads through its API, its events, or a copy built from them.

How this hook works: what "one writer, many readers" means

What the slogan means, phrase by phrase

"One writer": A single owning service writes each piece of data.
"many readers": Any number may read it, through the owner's API, its events, or read models.
"never through the back door": No reading or writing the owner's tables directly.

Why this picture fits: it covers ownership, CQRS and reporting with one rule.

The story → the concept

Flatmates	Distributed data
Your own cupboard	Database per service
Asking before borrowing	Using the owner's API
The shared shopping list	An event-fed read model
Reorganising a cupboard bit by bit	Lazy data migration

The trap it saves you from

Microservices that all reach into each other's tables: a distributed monolith held together by SQL.

7. Common mistakes

Reporting through the back doorA reporting job reading every service's tables freezes every schema. Feed reporting with events.

CQRS everywhereSeparate read and write models are worth it for busy, varied reads. For a simple CRUD screen they're just extra code.

Lazy migration with no endRows nobody reads never convert, so the old format lives forever. Add a background job to finish the tail.

Shared ownership by default"Both services update it" with no owner means conflicting rules and no one to ask.

8. Design exercise (30 minutes)

Find the owners (10 min). List the main tables of a system you know. For each, name the one service that should write it. Mark any with two writers.
Feed a report (10 min). Pick one cross-service report. Design the events it needs and the reporting table they build.
Plan a migration (10 min). For one schema change you'd like to make, write the expand-and-contract steps and how long each phase would run.

9. Self-check quiz

How many services should write a given table?

One owner keeps the rules for that data in one place.

What's the main cost of CQRS?

Reads are eventually consistent with the write model.

Which reporting option keeps services' data private and is close to real time?

Owners publish; reporting builds its own tables.

What's the weakness of pure lazy migration?

The cold tail stays in the old format; add a background job.

Inventory writes stock counts and Catalogue writes descriptions, in one products table. Best fix?

Each service becomes the single writer of its own data.

10. Go deeper

Resource	Time	Why
CQRS (Martin Fowler)	10 min	What it is, and the warning to use it sparingly.
Software Architecture: The Hard Parts	book	Data ownership, joint ownership options and breaking up shared databases.
Parallel Change (expand and contract) (Martin Fowler)	10 min	Changing an interface or schema without a big-bang release.