System design · Level 0: Basics · Module B6 of 6

From one server to millions of users

No large system started large. It started as one program on one machine, and every part you'll learn in this course was added later to fix a specific problem that appeared as users grew. Walking through that growth, one problem at a time, is the fastest way to understand why each building block exists, and it's how interviewers expect you to think: a real problem first, then the part that fixes it.

1. The idea

A food stall that becomes a restaurant chain. At first one cook takes orders, cooks and keeps the ingredients under the counter. When the queue grows, the pantry moves to a store room, more cooks are hired and a host seats people at whichever station is free. Popular dishes are prepared in advance, a warehouse supplies every branch, and finally branches open in other cities. Each change fixed one problem the stall actually had.

Growth puts pressure on four things, and every step below relieves one of them:

Capacity: one machine can only do so much work per second.
Reliability: one machine is one thing that can break and take everything down.
Latency: users far away, or slow work done while they wait, make the app feel slow.
Data: more data and more writes than one database can hold or absorb.

The two ways to add capacity have names you'll hear constantly. Scaling up (vertical scaling) means a bigger machine: more CPU, more memory. It's simple, but there's a largest machine, and it's still one machine. Scaling out (horizontal scaling) means more machines sharing the work. It has no hard ceiling, but the software must be designed for it: that is most of system design.

2. How it works

Press Play to grow a photo-sharing app from its first user. Each step names the problem that appeared and the part that fixes it.

Each step: what it fixes, what it costs

Step	Problem it fixes	What it costs	Module
1. One server	Nothing yet: the simplest thing that works	One failure takes everything down	B1
2. Separate database	Code and database fighting over one machine's CPU and memory	A network hop on every query; two machines to run	11
3. Scale up	Running out of CPU or memory, quickly	Big machines cost more per unit; there's a ceiling; still one machine	02
4. Load balancer + stateless app servers	One app server can't keep up, and is a single point of failure	Sessions must move out of servers; the balancer itself must be redundant	08
5. Database replicas	The database is a single point of failure; reads overload it	Replication lag: a replica may be slightly behind	14
6. Cache	The same popular data is read over and over	Stale data and invalidation; another system to run	09
7. Blob storage + CDN	Photos and videos are huge; distant users wait	Cache purging; storage and transfer bills	10
8. Queue + workers	Slow work (resizing photos, sending email) makes users wait	Work is "done later": retries, duplicates, monitoring	18
9. Monitoring and automation	You can't fix what you can't see, and many machines can't be managed by hand	Tools and time; alert fatigue if done badly	25, 27
10. Sharding	More writes, or more data, than one database primary can take	Cross-shard queries and transactions get hard; rebalancing	15
11. Multiple regions	Users worldwide; a whole data centre can fail	Data replication across the world; consistency choices; cost	27
12. Split into services	Many teams stepping on each other in one codebase	Network calls instead of function calls; much more to operate	28

3. Try it yourself

How big is big?

Slide the number of daily active users. The lab estimates the busiest second, works out which parts the app needs, and how many of each. Then pretend you skipped a step and see what breaks.

Illustrative model: each user makes 30 requests a day, the busiest second is 3× the daily average, 90% of requests are reads. One server running code and database together handles about 100 requests a second; one app server 1,000; one database primary 5,000 reads and 2,000 writes a second; a cache serves 90% of reads. Real numbers vary a lot; Module 03 shows how to estimate your own.

4. Trade-offs

Scale up vs scale out

Scaling up is a configuration change: no code changes, no new failure modes. Do it first. Scale out when you approach the biggest affordable machine, or when you need to survive one machine failing, which is usually sooner.

Simple now vs ready for later

Each step adds machines, bills and ways to fail. Add it when a number or a requirement demands it, not because a famous company uses it. A design for 10,000 users that's ready for 100 million is a design that's slow to build and hard to run.

Read scaling vs write scaling

Reads scale cheaply: caches, replicas and CDNs all copy data. Writes are harder, because every copy must agree; sharding is the main tool, and it's the most expensive step on the list.

Fewer, bigger parts vs many small ones

A modular monolith with one database goes remarkably far. Splitting into services helps large organisations move independently, and costs network calls, partial failures and operational load.

5. In real systems

Most products never need the last steps. Stack Overflow, one of the busiest sites on the web, described serving about 66 million page loads a day in 2016 from 9 web servers and 4 SQL Server machines in two clusters, with heavy caching. Good caching and a well-tuned database go a very long way.

The order is not fixed. A video app needs blob storage and a CDN on day one; a banking app needs replicas and backups before it needs a cache; a chat app needs stateful connection servers early (Module 19). The steps are a menu, chosen by the problem in front of you.

Say it in the interview

Two-sentence answers

"Vertical vs horizontal scaling?" Vertical means a bigger machine: simple, but capped and still a single point of failure. Horizontal means more machines: no ceiling, but the software must be stateless or partitioned to use them.
"How would you scale this to 10× the users?" Find the part that saturates first, with numbers; for read-heavy systems that's usually the database, fixed with a cache and replicas, and for write-heavy ones it's sharding.
"Why not start with microservices and sharding?" Because every part costs time and reliability. I'd start simple, measure, and add each part when a number tells me to.

Red flags interviewers notice

Drawing the 100-million-user architecture for a system with 10,000 users.
Adding servers behind a load balancer without making them stateless.
"Just add more servers" when the database is the bottleneck.

6. Remember it

Memory hook: "Problem first, part second"Never add a box without naming the problem it fixes, and the number that showed the problem.

How this hook works: what "Problem first, part second" means

What the slogan means, phrase by phrase

"Problem first": A real pressure: capacity, reliability, latency or data, ideally with a number ("reads hit 8,000 a second").
"part second": The building block that relieves exactly that pressure, with its cost said out loud.

Why this picture fits: it's how every real system grew, and it stops the two classic mistakes: adding parts with no reason, and adding the wrong part for the actual bottleneck.

The story → the concept

Food stall becoming a chain	System
Pantry moved to a store room	Separate database
More cooks, a host seating guests	Stateless app servers behind a load balancer
Popular dishes prepared in advance	Cache
A warehouse supplying every branch	Blob storage and CDN
Prep work done in the back	Queue and workers
Branches in other cities	Multiple regions

The trap it saves you from

A design with every famous component and no explanation for any of them.

7. Common mistakes

Scaling the wrong tierAdding app servers when the database is saturated makes things worse: more servers, more database connections. Find the bottleneck with numbers first.

Skipping statelessnessA load balancer in front of servers that keep sessions in memory gives random logouts. Make servers stateless before adding copies.

Sharding too earlySharding is the most expensive step. Scale up, add replicas and a cache first; shard when writes or data size truly exceed one primary.

Forgetting the single points of failureOne load balancer, one primary database, one region: each must have a plan for when it fails, even if the plan is "accept the risk".

8. Design exercise (30 minutes)

Grow a URL shortener (15 min). Draw it at 1,000, 1 million and 100 million daily users. At each size, name the part that saturates first and the step that fixes it.
Reorder for a different app (10 min). Write the order in which a video-streaming app and a payments app would need the twelve steps, and why they differ.
Check with the lab (5 min). Set the slider to your three sizes and compare.

9. Self-check quiz

What must you do before putting several app servers behind a load balancer?

Otherwise a user's next request lands on a server that doesn't know them.

Your database is overloaded by reads of the same popular items. What helps first?

Reads scale cheaply by copying data; sharding is for writes and size.

Why not just keep buying a bigger machine?

Scale up first because it's simple, but it neither removes the single point of failure nor grows forever.

Uploading a photo takes 8 seconds because the server makes five resized copies first. What fixes it?

Accept the upload, return at once, and let workers make the copies.

What should come before adding any new component to a design?

Problem first, part second.

10. Go deeper

Resource	Time	Why
Scale from zero to millions of users (Alex Xu, ByteByteGo)	30 min	The classic chapter that tells this same story, step by step.
Stack Overflow: the architecture, 2016 edition (Nick Craver)	30 min	How far a few well-run servers and caching can go.
The System Design Primer	browse	Short notes on every building block, with diagrams.