Prime Video: from serverless to one process
In March 2023 an engineer on Amazon Prime Video's video quality analysis team described how they rebuilt a stream-monitoring service. The first version was built from serverless parts orchestrated by a workflow engine. It hit a hard scaling limit at about 5% of the load it needed, and it cost too much to run at scale. Moving every component into a single process cut infrastructure cost by over 90% and let it monitor thousands of streams. The post was widely retold as "Amazon abandons microservices"; it says something narrower and more useful.
1. The situation
Prime Video streams live events to millions of viewers, and the team wanted to detect quality defects automatically: frozen video, block corruption, audio and video out of sync. The service has three parts: a media converter that turns a stream into frames and audio buffers, defect detectors that each analyse those frames for one kind of problem, and orchestration that controls the flow for each stream.
The first version was built quickly from managed serverless parts: AWS Step Functions orchestrated each stream, the converter uploaded frames to an Amazon S3 bucket, and each detector ran as AWS Lambda functions that downloaded the frames from S3.
2. What mattered
- Scalability: monitor every stream customers watch, thousands at once, not only the most popular ones.
- Cost: the post says the overall cost of the building blocks "was too high to accept the solution at a large scale".
- Evolvability: the team adds new detectors regularly.
- Speed of building had mattered most at the start, and the serverless design delivered it.
The first version failed on the first two. The post names two causes: Step Functions performed several state transitions for every second of every stream, which reached account limits and is billed per transition; and passing frames between components through S3 meant a high number of S3 requests, which was expensive.
3. The options
| Option | Good | Bad |
|---|---|---|
| Fix each bottleneck | Keeps the design; the team first considered this | Orchestration and data hand-offs are the design: every fix fights the per-operation pricing |
| Keep services, change the plumbing | Detectors still scale independently | Frames would still cross the network between processes on every hop |
| One process per stream group | Frames passed in memory; orchestration is plain code; runs on EC2/ECS, where compute savings plans apply | Detectors scale only vertically, within one instance; adding a detector is a deployment of the whole service |
4. The decision
The team chose the third option. Written as an ADR, it would read:
ADR: Run the monitoring pipeline as one process per instance
Status: Accepted
Context: Step Functions makes several state transitions per second of each
stream and reached account limits at ~5% of expected load; frames passed via
S3 cause a high, billed request volume. Components are tightly sequenced and
exchange large data (frames) many times a second.
Decision: Run converter, detectors and orchestration in one process on ECS
(EC2). Pass frames in memory. When detectors exceed one instance, run several
copies of the service, each with a subset of detectors, behind a lightweight
layer that distributes requests.
Consequences:
+ No per-transition or per-request charges on the hot path; savings plans apply
+ Simpler orchestration; same components and most of the code reused
- Detectors scale vertically; capacity per copy is bounded by one instance
- Media conversion is repeated in each copy, by choice (cheaper than caching it)
Revisit: if a detector needs independent scaling or a separate release cycle5. How it works
One stream through the first design, then through the second. Box text shows where each frame goes and what is billed.
6. The numbers
The post gives the result (over 90% lower infrastructure cost) but not the inputs. This lab rebuilds the shape with AWS list prices and invented workload figures: per frame, the first design pays one S3 upload, one download per detector, a Lambda run per detector and several workflow transitions; the second pays for instance time.
Illustrative assumptions, not Prime Video's figures: each frame makes (detectors + 2) Step Functions Standard transitions at $0.000025 each; one S3 PUT ($0.005 per 1,000) and one GET per detector ($0.0004 per 1,000); a 1 GB Lambda running 200 ms per detector per frame ($0.0000166667 per GB-second plus $0.20 per million requests). In the second design, one instance costing $0.17 an hour processes 40 detector-frames a second.
7. What happened
The post reports that the move cut infrastructure cost by over 90% and increased scaling capacity: the service handles thousands of streams with room to grow, and it can monitor every stream customers watch, not only the most viewed. Running on EC2 and ECS also let the team use compute savings plans. Because the high-level architecture stayed the same (the same converter, detectors and orchestration), much of the code was reused and the migration was quick.
Two details are easy to miss. When the detectors outgrew one instance, the team cloned the whole service, giving each copy a different subset of detectors: scaling by partitioning the work, not by splitting the process. And they chose to repeat the expensive media conversion in each copy rather than run it once and share the result, because sharing it turned out to cost more.
8. Lessons
- Logical and physical architecture are separate decisions. The components didn't change; only how they were deployed did (Module 01). Good component boundaries made the move cheap.
- Distribution has a price per hop. Parts that exchange large data many times a second, in a fixed sequence, and scale together are one architecture quantum. Splitting them buys independent scaling they didn't need and pays for every hand-off (Module 10).
- Per-operation pricing scales with chattiness, not with value. Serverless was right for building fast; at high, steady volume, per-transition and per-request charges overtook instance time (Module 16a, 22a's compute lab).
- It's one service, not a verdict. The post's own conclusion is that microservices and serverless "do work at high scale" and the choice must be made case by case. Reading it as "monoliths win" is the same mistake as the original design, in reverse.
9. Your turn
- Find the hot path (10 min). In a system you know, list each network hop or managed-service call that happens per request or per event. Which are billed per operation?
- Draw the quanta (10 min). Which components always scale together and exchange data in a fixed sequence? Would they be cheaper as one deployable unit?
- Write the revisit trigger (5 min). For the Prime Video ADR, what would make you split the detectors out again?
- "Prime Video went back to a monolith. Should we?" They merged one service whose parts passed frames to each other many times a second through a workflow engine and S3, both billed per operation. Merge parts that form one quantum like that; keep separate the ones that need to scale or release independently.
10. Check yourself
According to the post, what were the two main causes of cost and scaling problems in the first design?
How did the team scale once the detectors no longer fitted in one instance?
Why was the migration quick?
11. Sources and further reading
| Source | Why |
|---|---|
| Scaling up the Prime Video audio/video monitoring service and reducing costs by 90% (archived) | The original post: every fact in sections 1, 2, 4 and 7. |
| AWS Step Functions pricing | Per-transition pricing used in the lab. |
| Amazon S3 pricing | Request prices used in the lab. |