Design a Notification System

Many product teams need to reach users: one-time login codes, order shipped updates, password resets, "someone replied to you", and weekly marketing campaigns to tens of millions of people. Today each team integrates with push, email and SMS providers on its own, and users complain about duplicates, messages at 3 a.m. and ignored unsubscribes.

Design a shared notification platform. Teams send one request naming a user, a template and some data; the platform decides which channels to use based on the user's preferences, renders the message, delivers it through external providers, retries failures, and records what happened.

In 45 minutes, cover the API, how preferences and templates are applied, how messages flow to each channel, how you guarantee no lost and no duplicate sends, how urgent messages avoid getting stuck behind a big campaign, and how you respect provider and per-user rate limits.

Sketch on the HLD board, write your answer in the Design doc, and size it with the Estimator. Start the mock interview from the timer for a 45-minute run.

Design Path · Core Building Blocks · problem 5 of 70

Ask before you design. Try answering each yourself, then compare.

1. Which channels?
   Typical answer

   Mobile push (iOS and Android), email and SMS. In-app inbox is a nice-to-have.

2. What volume?
   Typical answer

   About 500 million notifications a day, with campaign peaks around 60,000 per second.

3. What delivery guarantee?
   Typical answer

   At least once internally, with deduplication so users effectively get each notification once. Some loss is acceptable only for low-priority marketing.

4. How fast must urgent messages arrive?
   Typical answer

   Login codes and security alerts should reach the provider within 2 seconds, even during a campaign.

5. Who controls opt-outs?
   Typical answer

   Users set preferences per category and channel; legal unsubscribes must always be honoured. Transactional security messages can bypass marketing opt-outs.

6. Do we need scheduling and quiet hours?
   Typical answer

   Yes: send-at times, and non-urgent messages wait for the user's local daytime.

7. Do we track opens and clicks?
   Typical answer

   Delivery status yes; opens and clicks are a follow-up.

Requirements

Functional

• Send a notification to a user (or a segment) by template and category, with optional channel override and send-at time.
• Apply user preferences, unsubscribes, quiet hours and per-user frequency caps.
• Render localized templates per channel.
• Deliver via push, email and SMS providers with retries and fallback.
• Expose delivery status per notification.

Non-functional

• Urgent notifications handed to the provider within 2 s p99, even under campaign load.
• No notification lost once accepted; duplicate sends under 0.01%.
• 500 M per day, 60k per second at peak.
• 99.95% availability for the send API.

Out of scope

• Building our own push, email or SMS delivery infrastructure (we use providers).
• Campaign authoring UI and audience analytics.
• Open and click tracking.

Back-of-the-envelope estimates

• Throughput: 500 M per day ≈ 5,800 per second on average. Campaigns are bursty, so plan for ~60k per second, 10× average.
• Channel mix: roughly 70% push, 28% email, 2% SMS. SMS is only 10 M a day but costs real money per message, so dedupe and caps matter most there.
• Delivery log: 500 M rows × ~1 KB (ids, channel, status history, rendered metadata) ≈ 500 GB a day; keep 30 days hot (~15 TB, ~45 TB replicated) and archive the rest.
• Preference lookups: one per notification, 60k/s at peak; 100 M users × ~500 bytes of preferences ≈ 50 GB, which fits in a cache cluster.
• Provider limits: a push provider may accept tens of thousands of messages per second per connection pool; SMS gateways often allow only a few hundred per second per account, which bounds how fast campaigns with SMS can go.

Bursty volume and external provider limits drive the design: queues absorb spikes, and per-provider throttling decides drain speed.

API

POST /v1/notifications
  Idempotency-Key: order-981-shipped
  {userId | segmentId, template: "order_shipped", category: "transactional",
   priority: "high"|"normal"|"bulk", data: {...}, channels?: [...], sendAt?}
  -> 202 {notificationId}
GET  /v1/notifications/{id}          -> {status per channel, attempts}
PUT  /v1/users/{id}/preferences      {category: {push, email, sms}, quietHours, tz}

The API returns 202 after the request is durably queued, not after delivery, and requires an idempotency key so caller retries never create a second notification.

Data model

• notifications(notification_id PK, idempotency_key UNIQUE, user_id, template, category, priority, data, send_at, created_at) in a sharded relational or wide-column store, partitioned by notification_id.
• deliveries(notification_id, channel, status, attempts, provider, provider_msg_id, updated_at) in a wide-column store, write-heavy and append-friendly, with a TTL of 30 days.
• preferences(user_id PK, per-category channel flags, quiet_hours, tz, unsubscribed_at) in a relational database with a cache in front; read on every send, changed rarely.
• devices(user_id, device_token, platform, last_seen) and contacts(user_id, email, phone, verified).
• templates(name, locale, channel, version, body), versioned so in-flight messages render with the version they were created with.

High-level design

A product service calls the notification API. The API checks the idempotency key, stores the notification row and publishes it to an intake topic in one transaction (via an outbox table), then returns 202. Scheduled and quiet-hours messages are parked with the scheduler, which republishes them at the right time; segment sends are expanded by the scheduler into per-user jobs in batches.

The notification service consumes intake, loads the user's preferences and contacts (cached), drops channels the user opted out of, enforces per-user frequency caps, renders the template per channel and locale, and publishes one message per channel to the channel topics. Each topic has separate high-priority and bulk partitions so login codes never wait behind a campaign.

Channel workers consume their topic, apply a per-provider rate limit, and call the push service, email provider or SMS gateway. Successes and provider message IDs go to the delivery log. Transient failures go to a retry queue with exponential backoff and jitter; after the final attempt the message lands in a dead-letter queue and, for urgent messages, the worker may fall back to another channel. Provider webhooks for bounces and delivery receipts update the same log.

Open the HLD board and press Reference to see the architecture diagram.

Deep dives

No lost and no duplicate sends

Losing a notification happens when the API acknowledges before the message is durable; fix that by writing the notification row and an outbox row in one database transaction, with a relay publishing the outbox to the queue. Everything downstream is at-least-once, so duplicates are the real risk. Give each delivery a deterministic key, notification_id:channel. Before calling a provider, a worker records the key as sending with a conditional insert; a redelivered message whose key is already sent is skipped. Pass the key to providers that accept idempotency tokens. A crash between the provider call and the status write can still duplicate, which is why the target is under 0.01% rather than zero.

Priorities and campaign fan-out

A campaign to 40 million users enqueued at once would block login codes for minutes. Separate traffic by priority: high, normal and bulk get their own topics and their own worker capacity, so a full bulk queue never delays high priority. Expand segments lazily: the scheduler pages through the audience in batches of ~10k users and enqueues the next batch only when bulk lag is below a threshold, which also lets a bad campaign be paused or cancelled midway. Spread a campaign across time zones so quiet hours are respected and provider limits are not exceeded.

Preferences, caps and quiet hours

Check preferences at send time, not at request time, because a user may unsubscribe between scheduling and sending. Read them from a cache with a short TTL and invalidate it on change; legal unsubscribes are written through so they apply immediately. Frequency caps (e.g. at most 3 marketing pushes a day) are counters per user, category and day in an in-memory store, incremented atomically before rendering. Quiet hours compute the user's local time; non-urgent messages are re-scheduled to the next allowed window instead of dropped. Security messages carry a category that bypasses marketing opt-outs but never a legal block.

Retries, provider limits and failover

Classify provider errors. Invalid tokens and hard bounces are permanent: mark the device or address invalid and never retry. Timeouts, 429s and 5xx are transient: retry with exponential backoff and jitter (1 s, 4 s, 16 s, up to about an hour), then dead-letter. Each provider gets a token bucket sized to its contract so workers do not trigger throttling, and a circuit breaker that trips on rising error rates. With a second email or SMS provider configured, an open breaker shifts traffic to the backup. Push tokens that a provider reports as unregistered are deleted to keep future sends cheap.

Failure modes

• If an SMS provider goes down, its circuit breaker opens, messages fail over to the backup provider or wait in the retry queue; OTPs may fall back to email.
• If the preference cache is unavailable, read preferences from the database at reduced throughput and pause bulk sends; never send marketing without a preference check.
• If a campaign floods the bulk queue, high-priority traffic is unaffected because it has its own topic and workers.
• If a worker crashes mid-send, the message is redelivered and the delivery-key check prevents most duplicates.
• If a bad template ships, version pinning and a per-template kill switch stop it; queued messages can be dropped by template version.

Trade-offs

• Accepting requests asynchronously (202) absorbs bursts and survives provider outages but means callers learn delivery status later.
• Separate queues per priority protect urgent messages but leave capacity idle when bulk traffic is quiet.
• Checking preferences at send time honours late opt-outs but costs a lookup per message instead of one per request.
• Aggressive retries improve delivery but risk duplicates and annoy users when the first attempt actually succeeded.
• Multiple providers per channel add resilience and price leverage but double integration, template and compliance work.

What a strong answer shows

• Accepts requests asynchronously with idempotency keys and durable queuing.
• Separates urgent and bulk traffic so campaigns cannot delay security messages.
• Has a concrete deduplication mechanism downstream of at-least-once queues.
• Applies preferences, caps and quiet hours at send time.
• Handles provider limits, error classes, retries with backoff and failover.

Mistakes to avoid

• Calling providers synchronously inside the API request.
• One queue for everything, so a campaign blocks password resets.
• Retrying permanent failures such as invalid phone numbers or uninstalled apps.
• Ignoring idempotency, causing duplicate SMS charges and annoyed users.
• Checking opt-outs only when the request arrives, missing later unsubscribes.

Follow-ups to expect

• How would you add an in-app inbox? (Write each notification to a per-user feed store and push a real-time signal over an open socket.)
• How would you batch noisy notifications into a digest? (Hold low-priority events per user for a window and render one summary.)
• How do you measure whether notifications are useful? (Track opens and clicks via redirect links and opens via provider callbacks, then tune caps.)
• How would you support multiple regions with data residency rules? (Route by user home region; keep contacts and logs in-region.)

Practice next