Design live comments

A cricket final is streaming live to five million people, and thousands of them are typing at once. Every viewer should see new comments within a couple of seconds, scroll back through what they missed, and never see abuse that moderators removed. The difficulty isn't storing comments. It's delivering one stream of them to millions of open connections, and deciding what a human can actually read.

1. The prompt

"Viewers of a live video can post comments. Everyone watching sees new comments appear in near real time. Design it."

The write side is modest; the read side is a fan-out problem. A comment written once may need to reach millions of screens within two seconds, over connections that drop whenever a phone goes into a tunnel. A good answer separates storing comments, routing them to the right servers, and delivering what a person can read.

2. Frame the problem

Ask these in the first five minutes. Each answer changes the design; guess before you open it.

3. What it must do

1. A viewer can post a comment on a live stream.
2. Viewers see new comments as they arrive, without refreshing.
3. A viewer can load earlier comments by scrolling back, during and after the stream.

Not now (with a reason):

• Replies and threads: they change the data model and the UI, and aren't needed for a live chat.
• Individual reactions: counted and shown as totals instead.
• Search over comments: a separate indexing job.
• Translation: an optional client feature on top.

4. How well it must do it

5. Numbers that change the design

Skip estimating total storage for years or API calls for history. They're ordinary, and nothing in the design depends on them.

6. The nouns

• Stream: a live video, with its status (live or ended) and owner.
• Comment: text by a user on a stream at a moment.
• User: with ban status.
• Connection: one open viewer session on a connection server.
• Moderation action: a removal or a ban.

comments  partition (stream_id, minute_bucket) · sort comment_id (time-ordered, e.g. Snowflake)
          user_id, text, video_offset_ms, status (visible | removed), score
streams   stream_id, owner_id, status, started_at, viewer_count (approximate)
bans      (stream_id or global, user_id) → until

7. The contract

Posting is an ordinary REST call. Live delivery is one-way from server to viewer, so Server-Sent Events fit: plain HTTP, built-in reconnection, and the browser resends the last event id it saw, so the server can resume. WebSockets work too, and are the choice if the same connection must also carry chat typing indicators or other client-to-server traffic. The user always comes from the session token.

POST /v1/streams/{stream_id}/comments          {text, client_id: "c-4f1…"}
     → 201 {comment_id, created_at}             (client_id dedupes a retried post)
GET  /v1/streams/{stream_id}/comments?before={comment_id}&limit=50   → older comments (scrollback)
GET  /v1/streams/{stream_id}/live               Accept: text/event-stream
     ← id: 7244031  event: comments  data: [{id, user, text}, …]     (a batch about once a second)
     ← id: 7244032  event: removed   data: ["7243990"]
     ← event: stats data: {viewers: 4812330, hearts_per_s: 9100}
DELETE /v1/comments/{comment_id}                (author or moderator)

Data flow: (1) a comment is checked and accepted → (2) stored, and appended to the stream's log partition → (3) routed to the connection servers that hold viewers of that stream → (4) each server batches, samples and sends once a second → (5) viewers that reconnect resume from their last event id.

8. First working version

Start with an API service, one database and a few connection servers. Get all three requests working, and note what won't scale.

8.1 Posting a comment

The API checks the user isn't banned, rate-limits them (say 1 comment every 2 s per user per stream), assigns a time-ordered id and writes the comment. The client_id makes a retried post (the phone lost the response) return the same comment instead of a duplicate.

8.2 Watching live (version one)

The viewer opens an SSE connection to a connection server. In this first version, the server asks the database once a second for comments newer than the last one it sent to each viewer, and pushes them.

8.3 Scrolling back

A history request reads the newest partition (stream_id, current minute) backwards from the given id, then moves to earlier minute buckets until it has 50 comments. A CDN can cache history pages of ended streams, because they never change.

9. Make it hold up

9.1 Getting each comment to the right servers fresh · scale

The channel layer can be Redis pub/sub (fast, fire-and-forget) or a log such as Kafka (replayable, with retention). A common split: Kafka partitions hold the durable comment log; a lightweight pub/sub tier does the live fan-out to connection servers; the log is the source for recovery.

9.2 The five-million-viewer stream fresh · light on phones

During a big moment, a top stream can receive 500 or more comments a second. Delivering them all would cost billions of messages a second, and no human could read them anyway. The answer is product and engineering together:

• Batch: each connection server sends one message per viewer per second containing that second's comments. Messages per second become viewers × 1 instead of viewers × comments.
• Sample by relevance: when a second has more comments than a viewer can read (say more than 10), choose which to show. Prioritise the creator and moderators, the viewer's friends, verified accounts and high-quality comments (a score from the classifier), then fill with a random sample. Every server can apply the same rules.
• Personal first: a viewer always sees their own comment immediately, inserted by their client, even if the sample left it out for others.
• Reactions as counts: hearts are counted per second per server and summed; viewers receive "9,100 hearts/s" and the client animates floating hearts locally.

The lab in section 13 shows the difference on a phone screen.

9.3 Storing comments without a hot partition kept · scale

9.4 Ordering, reconnects and duplicates fresh · kept

• Time-ordered ids: comment ids embed a timestamp (a Snowflake-style id, Module 20), so sorting by id is sorting by time. No global sequencer is needed, and two comments a second apart may swap, which the requirements allow.
• Batch ids for resume: each live message carries an SSE id. When a phone reconnects, the browser sends Last-Event-ID automatically, and the server replays what was missed from a short in-memory buffer of recent batches per stream (the last two minutes). Longer gaps fall back to a history read.
• Dedupe on the client: the client keeps the ids it has shown, so a batch replayed after a reconnect doesn't show comments twice.
• Reconnect politely: clients retry with exponential backoff and jitter. The server can send an SSE retry: value to spread reconnections after an incident.

9.5 Moderation that keeps up safe

Also give creators their own moderators, slow mode (a longer per-user rate limit), "followers only" mode, and shadow bans (the banned user still sees their own comments, so they don't simply create a new account).

9.6 Running the connection servers scale · surviving failures

• Stateful but disposable: a connection server holds open connections and its channel subscriptions, nothing else. If it dies, its viewers reconnect elsewhere and resume from their last event id.
• Tuning for many connections: event-driven servers, raised file-descriptor limits, small per-connection buffers, idle heartbeats every 15–30 s so proxies don't drop quiet connections. LinkedIn has written about serving hundreds of thousands of SSE connections per machine this way.
• Draining on deploy: never restart 50,000 connections at once. Stop accepting new connections, then close existing ones gradually over a few minutes with a retry hint, so the reconnections spread out.
• Load balancing: balance on connection count, not request rate. Route by stream_id so viewers of the same stream share servers, with a group of servers per huge stream.

9.7 The whole design after the deep dives

Follow one comment through every part: checks, the log, storage, per-stream channels, connection servers that batch and sample, and the viewers' phones.

10. Interviewer follow-ups

Expect several of these after the main design. Give the mechanism and its cost in under a minute.

11. When things break

12. Wrap-up and the clock

Trade-offs we made:

• Sampling instead of "everyone sees everything" on huge streams.
• Rough time order instead of a global sequence.
• SSE for one-way delivery.
• Availability over exactness during failures.
• Holding comments for scoring only where the audience is large.

What we'd watch:

• Post-to-screen latency p99, measured by synthetic viewers.
• Connections per server.
• Comment rate per stream.
• Removal-to-screen time.
• Reconnects per minute.
• Bytes per viewer.

At 10×:

• Regional relay tiers.
• Larger server groups per stream.
• Stricter caps for the largest audiences.

Common mistakes:

• Polling the database per viewer.
• Promising every comment to every viewer of a 5M stream.
• One partition per stream with no time bucket.
• Forgetting reconnects on mobile.
• Treating moderation as an afterthought.
• Using push notifications for live delivery.

13. Try it, check yourself, read more

The comment firehose. The phone shows comments arriving at the rate this viewer would actually receive them. The numbers show what the servers must send.

Illustrative capacity: one connection server holds 50,000 viewers and sends up to 500,000 messages a second; a comment is ~250 bytes.