Skip to content

How to design a notification system

The first sections are free. The deep dives, level expectations and quiz are part of Pro.

On this page
Asked at
Practice this question

The problem

A notification system is the one service other teams call when they need to reach a user by push, text message or email. It takes a request from a calling service, checks what the user has allowed, fills in a template, and hands the message to a provider it does not run: APNs or FCM for push, a carrier gateway for SMS, a mail provider for email. The answer turns on that dependence. The volume is modest, about 185 sends a second on average, so throughput is not the hard part. The hard parts are keeping the other channels sending while one provider stalls, and making sure a timed-out send is recorded and retried rather than silently dropped.

Clarifying questions

AskWhy it mattersA typical answer

Who sends notifications, and how many a day on each channel?

It tells you whether the callers are internal services you can authenticate with a key, and whether throughput or isolation is the hard part. The split by channel also decides how many worker pools you draw.

Internal services and scheduled jobs call the API. About 10 million push notifications, a million text messages and 5 million emails a day, which is about 185 sends a second on average.

Is a late notification acceptable, and is a lost one?

The answer sets the delivery semantics. If a lost notification is worse than a repeated one, you design for at-least-once delivery with retries and accept rare duplicates.

Soft real time: fast normally, delays are fine under load. Losing a notification is not acceptable.

Do users control which channels reach them?

A per-channel opt-out means a settings table that sits on the send path, and a decision about where it is checked and how it is cached.

Yes. A user can opt out of each channel separately, and the system has to respect that before anything goes out.

Which devices and platforms do we support?

Each platform can mean another provider with its own token format, limits and error codes, and a user may have several devices.

iOS, Android and desktop. A user can own more than one device, and each device registers its own token.

Do urgent notifications, such as a password reset code, share the path with marketing sends?

If they do, a marketing blast can sit in front of a code the user is waiting for right now. Knowing early lets you add a priority lane and a transactional class that skips the per-user cap.

Both go through this service. Treat reset codes and other transactional sends as urgent and keep them from waiting behind bulk sends.

Does the service need to know whether a message was delivered, or just that the provider accepted it?

Provider acceptance is what the send path can observe directly; delivery status arrives later through callbacks or not at all, and tracking it means a status column that changes over time.

Record acceptance and failure per attempt in a log. Delivery callbacks and open or click tracking are a later addition for tuning volume.

The requirements menu

RequirementScopeWhy

Send push, SMS and email notifications, triggered by services or scheduled jobs

Core

The whole product. One API that any internal service or cron job can call, with the channel chosen per request.

Users can opt out per channel

Core

A product requirement and a design one: a settings table that is read on each send and must stay correct when cached.

Build notifications from reusable templates

Core

Callers pass a template id and parameters instead of a body, so wording, localization and the call to action live in one place and the API stays small.

Support iOS, Android and desktop

Core

In this design each provider (APNs, FCM, an email or SMS gateway) has its own tokens and error codes, which is why it has a queue and a worker pool per provider rather than per channel.

A transactional class of sends that bypasses per-user caps and takes a priority lane

Optional

Worth taking: it is the answer to the password reset code that must not wait behind a marketing blast, and it costs one field on the template and one extra queue.

Delivery status callbacks from providers written back to the notification log

Optional

Take it if time allows. Providers report some outcomes after the fact (bounces, dead tokens, and on some channels deliveries); recording them lets you drop dead devices and measure real delivery rather than acceptance.

Collapse or replace a pending push when a newer one supersedes it

Optional

Useful for scores and counters, and both APNs and FCM support it with a collapse id. Leave it unless asked; it adds a field, not a component.

Open and click tracking to tune volume

Optional

A later addition: workers emit events to an analytics service, and the numbers feed the per-user limits. Name it, park it.

Quiet hours and digests that batch low-priority sends

Optional

A real feature on top of a scheduler. Leave it unless the interviewer asks; the scheduled-jobs trigger already gives you the hook.

Building the providers themselves

Out of scope

APNs, FCM and carrier gateways are outside services. The design treats each as a dependency that can slow down or fail, nothing more.

In-app notification feeds

Out of scope

A feed is a read path with its own store and pagination. Say the same event could be written there, and stop.

Marketing campaign tools

Out of scope

Audience building and scheduling are a separate product that calls this API in bulk. The one thing to keep from it is that its bursts must not starve urgent sends.

Two-way SMS and email replies

Out of scope

Inbound messages need webhooks, parsing and routing to the right team. A different system.

Non-functional requirements

RequirementWhat it forces

Soft real time: fast normally, delays are fine under heavy load

The API answers as soon as the message is durably queued and does not wait for a provider. Queues absorb bursts instead of the servers, and the alarm to set is on the age of the oldest queued message, not on per-send latency.

No notification is lost

A durable queue that stores a message before acknowledging it, a log row per notification, retries with backoff, and a dead-letter queue that alerts a person. Delivery is at-least-once, with the event id used to catch the repeats this system itself creates, because at-most-once gives up on a timed-out send, and that send may be lost.

10 million push, 1 million SMS and 5 million emails a day

About 185 sends a second on average, with push at about 116 of them. That is a small load: one database with a replica holds the log and the settings, and the design effort goes to isolation and retries rather than sharding. Worker pools are sized per channel from this split.

An outage at one provider does not stop the other channels

A queue and a worker pool for each provider, short timeouts on provider calls, and no shared pool of threads that a stalled provider can fill. Monitoring is per queue, so an SMS backlog shows up as an SMS problem.

Estimates

QuantityWorkingResultWhat it sizes

Average sends a second, by channel

10 million push + 1 million SMS + 5 million email = 16 million a day. Divided by 86,400 seconds: push ≈ 116 a second, SMS ≈ 12 a second, email ≈ 58 a second, about 185 in total

About 185 sends a second: 116 push, 12 SMS, 58 email

The worker pool for each channel, and the observation that the load is small enough that isolation and reliability, not throughput, decide the design.

Concurrent provider calls each channel needs

As an example assumption (not a provider figure), say a provider call takes about 200 ms, so one worker thread completes about 5 sends a second. Push: 116 ÷ 5 ≈ 23 threads; email: 58 ÷ 5 ≈ 12; SMS: 12 ÷ 5 ≈ 3

About 23 concurrent push sends, 12 email and 3 SMS at the average rate: a few worker processes per channel

How many workers each queue needs to stay short, and how much headroom to add before a backlog forms.

SMS backlog after a one-hour provider outage

12 a second × 3,600 seconds ≈ 43,200 texts waiting. The usual SMS pool is about 3 threads at 5 sends a second each; tripled to 9 threads it sends about 45 a second once the provider returns. New texts keep arriving at 12 a second, so the net drain is 45 - 12 = 33 a second, and 43,200 ÷ 33 ≈ 1,309 seconds, about 22 minutes

About 43,200 queued texts, cleared in about 22 minutes with three times the usual workers

Queue retention, the worker headroom to add after an outage, and how long a reset code would wait unless it carries an expiry.

Notification log growth

Assume about 200 bytes a row (event id, user, channel, template, status, attempts, timestamps). 16 million rows a day × 200 bytes ≈ 3.2 GB a day; over 30 days ≈ 96 GB

About 3.2 GB a day, about 96 GB a month

Why one database with a replica holds the log, and why a store sharded across regions is more than about 185 sends a second needs.

A marketing blast compressed into one hour

If a campaign sends the whole day's 5 million emails in one hour: 5,000,000 ÷ 3,600 ≈ 1,389 a second, against the 58 a second average, about 24 times it

About 1,389 emails a second for an hour, about 24 times the average

The email queue depth during a blast, and why a reset code needs its own lane instead of a place in that queue.

Core entities

EntityWhat it holds

User

The person being reached, with the addresses the channels need.

Device

One row per installed app that can take a push; a user can have more than one device.

Notification setting

Whether a user accepts a channel; read before anything is queued, and cached.

Template

The wording of a notification with placeholders, filled in from the caller's params.

Notification log

One row per message per recipient and channel, written before the send and updated after it.

The API

EndpointNotes

POST /notifications { user_ids, channel, template_id, params, event_id } → { status }

In this design each calling service holds a key and secret pair and unsigned requests are rejected, which keeps sends to verified internal callers. The server validates the template and params, expands user_ids into one message per recipient, and checks opt-in. We'd have it answer 202 Accepted as soon as the messages are queued rather than after a provider replies. event_id is the caller's idempotency key: in this design the server looks it up in the log and does not queue a second copy for a repeat. Limits per caller and per user apply here, and we'd return 429 Too Many Requests for an over-limit send.

PUT /users/{id}/settings { channel, opt_in }

Writes the settings row, then removes the cached entry for that user and channel so the next send reads the new value. Workers re-check opt-in before sending anything that waited in a backlog, so an opt-out also covers messages already queued.

POST /devices { platform, device_token }

Called by the app when it installs or when the platform hands it a new token. Upserts the device row for the signed-in user. When APNs answers HTTP 410 for a token, the push worker marks that row disabled so later sends skip it.

The high-level design

settings, devicessendCalling serviceNotificationserversSettings DBProviders

Step 1 of 4

Servers that call the providers directly

Start with the simplest shape. Calling services post to notification servers behind a load balancer. The servers check the caller's key, read the user's contact details, devices and opt-in settings from the database, fill in the template, and call the provider for the channel: APNs or FCM for push, the SMS gateway, the mail provider. This already covers the four functional requirements. Its weakness is the provider call inside the request: the caller waits for an outside service, and a slow SMS gateway holds server threads that push and email requests also need.

app keyopt in?missper userurgentsendfailedbackoffgave upopens, clicksCalling serviceLoad balancerNotificationserversCacheSettings, devices,templatesRate limiterQueue perproviderPriority queueWorkers per queueProvidersLogRetry queueDead letters,alertAnalytics
The whole design

What breaks first

The deep dives

Never losing a notification. The SMS gateway stops answering while a password reset code is on its way out. Walk one notification from the calling service to the phone, say what happens to the code and to the push and email channels, and say whether a retry can deliver it twice.

A concept that helps here: Retries and dead-letter queuesA concept that helps here: Delivery semantics: at most, at least, exactly onceA concept that helps here: Idempotency keys

Opt-outs, rate limits and priority. A buggy calling service starts sending one user several notifications a minute on all three channels. Where is the opt-out checked, what stops the flood from reaching the phone, and how does a password reset code get through the same protection?

A concept that helps here: Rate limiting algorithmsA concept that helps here: Cache invalidation and TTLsA concept that helps here: Backpressure and load shedding

Concepts this guide uses

Sources

  1. Alex Xu, System Design Interview, chapter 11: Design a notification system (reading notes at books.dwf.dev)
  2. Apple: Local and Remote Notification Programming Guide, Communicating with APNs
  3. Firebase Cloud Messaging: Non-collapsible and collapsible messages
  4. Firebase Cloud Messaging: Understanding message delivery
  5. The Firebase Blog: Life of a message from FCM to the device
  6. Amazon SQS Developer Guide: Amazon SQS standard queues
  7. Amazon SQS Developer Guide: Amazon SQS queue types
  8. Amazon SQS Developer Guide: Using dead-letter queues in Amazon SQS
  9. Marc Brooker: Jitter: Making Things Better With Randomness
  10. OneSignal REST API reference: SMS (idempotency_key)
  11. Twilio Segment: Delivering billions of messages exactly once
  12. Microsoft Learn: Rate limiting middleware in ASP.NET Core (token bucket limiter)
Sign in to report a mistake