Design a notification system

Every product eventually needs to tell users things: a login code, "your order has shipped", a reminder, a weekly digest. A notification service takes those requests from many internal teams and delivers them through push, email, SMS and an in-app inbox, at the right time, once from the user's point of view, without spamming anyone, and without a marketing campaign delaying someone's password-reset code. It follows the interview framework phase by phase.

1. The prompt

"Design a system that sends notifications to users across channels. Internal services (orders, security, marketing) call it; it decides how and when to reach each user."

The hard parts are not the sending itself, which external providers do, but everything around it: separating urgent from bulk traffic, never sending twice, surviving providers that fail and throttle, respecting each user's preferences and time zone, and pushing a 20-million-person campaign without hurting anything else.

2. Frame the problem

3. What it must do

1. A service can send a notification to a user by type and data; the system picks channels, renders the template and delivers it.
2. The system respects each user's preferences: opt-outs, quiet hours, frequency caps.
3. A team can send a campaign to a large audience, optionally at a local time in each user's time zone, and see delivery status.

Not now:

• The audience-segmentation tool: we consume its output.
• The template editor UI: standard tooling.
• Two-way messaging: replies, chatbots.

4. How well it must do it

5. Numbers that change the design

6. The nouns

• Notification: one message to one user.
• Delivery: one notification on one channel, with its status.
• Preference.
• Device: a push token.
• Template.
• Campaign: a batch of notifications to a segment.

notifications  notification_id, user_id, type, priority, data, idempotency_key, created_at
deliveries     (notification_id, channel) → status (pending | sending | sent | delivered | failed), provider_msg_id
preferences    user_id → per (channel, type) opted_in, quiet_hours, time_zone, caps
devices        user_id → [platform, push_token, last_seen]
campaigns      campaign_id, segment, template, send_at_local, progress (last batch done)

7. The contract

REST for internal callers, with the calling service identified by its credentials. An idempotency key lets callers retry safely.

POST /v1/notifications            Idempotency-Key: order-881-shipped
{ "user_id": 42, "type": "order_shipped", "priority": "transactional",
  "data": {"order_id": 881, "eta": "Friday"} }
  → 202 Accepted {notification_id}

POST /v1/campaigns {segment_id, type, template, send_at_local: "09:00"} → 202 {campaign_id}
GET  /v1/notifications/{id}       → {status per channel}
PUT  /v1/users/{id}/preferences   (from the user's own settings page; user from their session)

Data flow: (1) accept and dedupe → (2) check preferences and choose channels → (3) render the template → (4) enqueue per channel and priority → (5) a channel worker calls the provider → (6) status events come back and update the delivery.

8. First working version

An API service, a database, one queue, and one worker pool that sends through each provider.

8.1 Sending one notification

The API stores the notification, checks preferences, renders the template for each allowed channel, and enqueues one message per channel. Workers take messages from the queue, call the provider, and record the result.

8.2 Sending a campaign

The campaign API loops over the segment and calls the same path for each user.

8.3 Showing status

Workers write "sent" to the deliveries table; provider webhooks update it to "delivered" or "failed".

9. Make it hold up

9.1 Login codes must not wait behind a campaign urgent is fast

9.2 Once, from the user's point of view once, not lost

9.3 Providers fail and throttle provider-proof

• Each channel worker wraps its provider (APNs for iOS, Firebase Cloud Messaging for Android and web, an email service, an SMS gateway) with timeouts, retries with backoff and jitter, and a circuit breaker.
• Keep a second provider for SMS and email, and fail over when the first's breaker opens.
• Respect provider rate limits with a token bucket per provider and account (Design D02), so a campaign slows down instead of getting the account blocked.
• Messages that fail permanently (invalid number, uninstalled app) aren't retried: they update the device or contact record. Expired push tokens are removed.

9.4 Preferences, quiet hours and frequency caps compliant · once, not lost

Before sending, check the user's preferences (cached, since they're read for every message):

• Opt-outs and unsubscribes: unsubscribing is legally required for marketing email.
• Quiet hours in the user's time zone: delay non-urgent messages until morning; never delay login codes.
• Frequency caps, such as "at most 3 marketing pushes a week", counted in a fast store.
• Active users: if the user is active in the app right now, an in-app message may replace the push.

When a user unsubscribes, the change event invalidates the cached preferences at once, so the next message respects it.

9.5 A campaign to 20 million users scale

Don't enqueue 20 million messages in one API call. A campaign job reads the segment in pages, creates batches (say 1,000 users each) as jobs on the bulk queue, and records progress, so a crash resumes where it stopped. For "9:00 in each user's time zone", group users by time zone and schedule each group's batches separately (a scheduler like Design D21 does this well). Throttle to the provider quota left over after transactional traffic.

9.6 Tracking delivery once, not lost

Statuses arrive asynchronously: provider responses ("accepted"), delivery receipts and webhooks ("delivered", "bounced"), and opens and clicks from tracking links or the app. Write them as events to a log, and update a per-delivery status record. Analytics (open rates per campaign) comes from the event stream, not from querying the status table. The in-app inbox is a per-user list in a database, read when the app opens, plus a real-time nudge over WebSocket or SSE if the app is open (Module 19).

9.7 The whole design after the deep dives

A login code and a campaign at the same moment, through the final design.

10. Interviewer follow-ups

11. When things break

12. Wrap-up and the clock

Trade-offs we made:

• Separate lanes per priority (more infrastructure, guaranteed latency for urgent messages).
• Delivery records for dedup (an extra write per send).
• Erring towards duplicates for login codes, and towards losses for marketing.
• Batched, resumable campaigns.
• Status from an event stream.

What we'd watch:

• Transactional time-to-provider p99.
• Queue depth per lane.
• Provider errors and 429s.
• Duplicate rate.
• Delivery and opt-out rates per type.

At 10×:

• Partitioned queues.
• More provider accounts.
• Local preference caches.

Common mistakes:

• One queue.
• Claiming exactly-once delivery.
• No plan for provider limits.
• Sending 20M messages from one request.
• Ignoring quiet hours and unsubscribes.
• Querying the status table for analytics.

13. Try it, check yourself, read more

At 9:00 a campaign of 600,000 messages is queued. Login codes keep arriving at 50 a second. Workers can send 2,000 messages a second in total. Compare one shared queue with separate queues.

Model: with one queue, messages are sent in arrival order. With separate queues, transactional messages get a reserved 10% of the capacity (and any unused share goes to the campaign).