Design a recommendation system

Design the "For you" feed of a large online marketplace: 200 million listings, 30 million shoppers a day, and a home page that should show each of them 50 things they're likely to want, in under 150 milliseconds. The same shape powers "up next" on video sites, playlists in music apps and "people also bought" on product pages: find a few hundred good candidates cheaply, rank them carefully, then make the list varied and fair.

1. The prompt

"Design the personalised recommendations on a marketplace's home page."

What makes it hard:

• Too many items to score. No model can score 200 million listings for every page view in 150 ms. The design is a funnel.
• Two clocks. Models train on months of data overnight, but a shopper who just searched for hiking boots expects boots on the next page.
• It learns from itself. People can only click what was shown, so yesterday's recommendations shape tomorrow's training data. Without care, new items never get a chance and the feed narrows.

2. Frame the problem

3. What it must do

1. Return a ranked, varied list of about 50 items for a shopper, quickly.
2. React within the session to what they view, search and add to their basket.
3. Learn continuously from what was shown and what happened next.
4. Handle the new: new shoppers with no history, and new listings nobody has seen.

Not now: search ranking (similar machinery, different input), email recommendations, sponsored placements (that's D41).

4. How well it must do it

5. Numbers that change the design

Illustrative numbers.

6. The nouns

• Shopper and item (a listing), each with an embedding: a vector where similar tastes and similar items sit close together.
• Event: an impression (shown at position 7), click, add to basket, purchase, return.
• Candidate: an item proposed by one retriever, with the retriever's name and score.
• Feature: a number the ranker uses, like "this shopper's purchases in this category, last 30 days".
• Model version: trained weights plus the exact feature definitions they expect.

events           (shopper_id, ts) → item_id, kind, position, surface, request_id      (stream → warehouse)
item_features    item_id → price, category, seller, stock, 7-day sales, embedding_v12     (online store)
shopper_features shopper_id → category affinities, price band, embedding_v12, last_50_views (online store)
co_bought        item_id → [(item_id, score)] top 200                                     (nightly)
ann_index        embedding_v12 → item_id                                                  (memory, sharded)
feed_log         request_id → model version, candidates, scores, final order              (for training and debugging)

7. The contract

GET  /v1/feed?surface=home&limit=50&cursor=…   → [{item_id, reason?}], request_id, next cursor
POST /v1/events   [{request_id, item_id, kind: impression|click|basket|purchase, position, ts}]   (batched by the app)
Internal: POST /rank {shopper_id, candidates[], context} → scores      (gRPC, same data centre)

The shopper comes from the auth token. The request_id ties every later click and purchase back to exactly what was shown and by which model version: without it, training data can't be trusted.

8. First working version

Start simple and still correct: a nightly job computes "bought together" lists and popular items per category; the feed service merges a shopper's recent views with those lists, filters, and returns. Then replace the merge with a retrieve → rank → re-rank pipeline, keeping the simple version as the fallback.

8.1 Serving one feed

8.2 The learning loop

9. Make it hold up

9.1 Retrieve cheaply, rank carefully speed

Each retriever is good at one kind of candidate, so several run in parallel, each returning a couple of hundred: item-to-item ("bought together" with what you viewed), embedding search (items near your taste vector, from a two-tower model), trending in your region and categories, and fresh items for exploration (9.4). Retrievers can be wrong cheaply, because the ranker sees everything they propose. Measure each one by how often it supplies what people end up buying.

9.2 Features that mean the same thing in training and serving correctness

9.3 Reacting within the session freshness

Three speeds, each cheap at its own job:

• Batch, nightly: train models, rebuild embeddings, co-bought lists and the ANN index.
• Near-line, seconds: a stream job reads click and view events and updates the shopper's online features ("last 50 views", category counts) and the session embedding (an average of recently viewed item embeddings).
• Online, per request: the retrievers query with the latest session embedding, and the ranker reads features fresh from the online store.

So the boots viewed a minute ago change which candidates come back and how they're scored, without retraining anything.

9.4 New items, new shoppers and feedback loops fairness

9.5 Knowing it works quality

• Offline first: replay last week's sessions and check whether bought items were in the top 50 (recall) and how high (ranking metrics such as NDCG). Cheap, fast, and only a filter: a better offline score often doesn't win online.
• Then an A/B test: 1–5% of shoppers get the new model; compare purchases per visit and the guardrails (returns, complaints, seller spread, latency) for two weeks, to cover weekly patterns.
• Roll out like code: a model version is deployed with its feature definitions, canaried, and can be rolled back in minutes (Module 27).
• Watch for drift: feature distributions and click rates by model version, daily.

9.6 The whole design after the deep dives

10. Interviewer follow-ups

11. When things break

12. Wrap-up and the clock

Trade-offs we made:

• A funnel (several cheap retrievers, one careful ranker, a rules-based re-ranker), accepting more moving parts for latency.
• Three speeds: nightly training, near-line features, per-request scoring.
• A feature store with point-in-time training data, to avoid skew and leakage.
• A small, explicit exploration budget, paid for in a little short-term click rate.
• Purchases with guardrails as the target, not clicks.

What we'd watch: p99 per stage · recall of each retriever · purchases per visit by model version · share of impressions going to new items · feature drift · stream lag.

At 10×: more ranker replicas (the per-request cost); a sharded ANN index; the batch side grows with data, not traffic.

Common mistakes:

• One model scoring the whole catalogue per request.
• Optimising clicks and calling it success.
• Training and serving features computed by different code.
• No plan for new items or new users.
• No request ID linking outcomes to what was shown.

13. Try it, check yourself, read more

The exploration lab. Five established items have years of clicks. A new item arrives with none. The feed shows the item with the best estimated click rate, except for a share of impressions spent exploring at random. Choose how good the new item really is and how much to explore; each setting is simulated 20 times over 20,000 impressions.

The established items' true click rates are 4.0%, 3.5%, 3.0%, 2.5% and 2.0%, each with 4,000 past impressions. Exploration here is "epsilon-greedy", the simplest bandit rule; real systems use smarter ones (Thompson sampling, upper confidence bounds).