Skip to content

How to design an ad serving system

The first sections are free. The deep dives, level expectations and quiz are part of Pro.

On this page
Practice this question

The problem

An ad serving system fills each ad slot in a feed with one ad, chosen from hundreds of thousands of live campaigns while the page waits for the answer. The design turns on doing the whole decision inside a fixed slice of time: narrow the candidates in memory before anything expensive runs, treat budgets and frequency caps as cheap local checks fed by events that arrive late, and return no ad rather than a slow one.

Clarifying questions

AskWhy it mattersA typical answer

How many ad requests arrive at peak, and how long may the server take?

The request rate decides how many servers hold the index and how much shared state they can touch; the time budget decides how many candidates the model may score.

Plan for 20,000 to 100,000 requests a second at peak, 100 milliseconds end to end with 50 to 70 of them on the server. If the interviewer moves to the streaming framing, 1 million a second with a decision in single-digit to low-tens of milliseconds, say what changes: the same shape, less room per step.

How many campaigns are live, and how soon must a change to one of them reach serving?

Several hundred thousand campaigns fit in each server's memory; the freshness requirement decides whether the index is rebuilt or patched with deltas.

Hundreds of thousands of active campaigns. A paused campaign or a new bid should stop or start serving within minutes; the campaign editing screens themselves are out of scope.

Is a small overspend acceptable, or must spend stop exactly at the budget?

An exact stop needs a write to shared state on each request; a tolerance lets spend be counted from events that arrive late.

A small single-digit percentage of overspend is allowed in the worst case. Spend and impressions must show up in reports within 5 minutes.

Do we pick from our own campaigns, or also call outside exchanges?

Calling an exchange adds a network hop with its own timeout and a second auction; it changes the time budget and the failure story.

First-party campaigns, nothing else. Real-time bidding exchanges are out of scope.

What does the feed show when no ad comes back in time?

If the feed can fill the slot with organic content, the serving path can fail open to no ad instead of making the page wait.

The feed renders organic content in the slot. Serving no ad is acceptable; a slow or failed page is not.

How strict are frequency caps, and who defines them?

A cap that may overshoot by one view can be checked from counts that arrive late; an exact cap needs an atomic reservation when the ad is served.

Advertisers set caps such as three views a day per user per ad or campaign. A small overshoot is tolerable by default; in this design, exactness is a per-campaign setting rather than the baseline.

The requirements menu

RequirementScopeWhy

For each ad slot request, return at most one ad, or none, from the eligible campaigns

Core

The whole product: one request in, one ad or an empty answer out. Returning none is a valid answer, which is what lets the path fail open.

Match targeting: audience segments, keywords and location

Core

Targeting is what makes retrieval possible: these keys become the index that cuts several hundred thousand campaigns down to some hundreds.

Rank eligible ads by bid and predicted click rate, and run an auction for the price

Core

Ranking is the expensive step, so where it sits in the funnel decides whether the time budget holds. The auction sets what the advertiser pays.

Spend each campaign’s budget evenly over its flight, without overspending

Core

Budgets are the money side of the system. Even spend needs a pacing signal; no overspend needs a story for spend events that arrive late.

Cap how often one user sees the same ad or campaign

Core

Caps need per-user state read on each request, which is the one piece of user-level data on the hot path.

Confirm that an impression was seen before counting it toward caps and spend

Optional

Worth taking: the API already has an impressions endpoint, and counting served ads that scrolled past unseen inflates caps, and spend too when the advertiser is billed for viewed impressions. Leave it if time is short; counting served ads is a stated approximation.

Fill several slots on one feed page without repeating an ad

Optional

Real feeds have several slots per page. Take it if asked: it means passing the ads already chosen for the page as an exclusion list on the next request, which is cheap. Otherwise park it.

Budgets at several levels: campaign, ad group, and per day

Optional

Advertisers set budgets at more than one level. It adds a check per level in the filters, with the tightest one deciding; mention it and add it when the interviewer wants the budget dive to go further.

A delivery report per campaign: spend, impressions and pace

Optional

The API has it, and advertisers watch it. It reads from the spend ledger the pacing service already fills, so it costs little to include; the full reporting pipeline is a separate question.

Filter bot and fraudulent impressions before they count

Out of scope

Invalid traffic matters to billing, but detection is its own system. Say that impression events carry a signature and can be marked invalid downstream, then move on.

Screens where advertisers create and edit campaigns

Out of scope

Ordinary create and update forms over the campaign table. The serving design needs just the table they write to and a change feed from it.

Training the click prediction model

Out of scope

An offline pipeline with its own design. The serving side needs the model's output and its latency, nothing about how it was trained.

Outside real-time bidding exchanges

Out of scope

A second auction over the network with its own timeouts. Naming the hop you would add is enough.

Counting clicks for advertiser reports, which is the ad click aggregator question

Out of scope

Click aggregation is a stream-processing question of its own. Here clicks matter as events that feed spend; the reporting side is parked.

Non-functional requirements

RequirementWhat it forces

An answer within 100 ms end to end, 50 to 70 ms of it on the server

Nothing on the serving path may wait on a database or a remote write. The index lives in memory on each server, spend and cap checks are local cache reads, and each step gets its own deadline so a slow model turns into a cheaper pick or no ad instead of a late page.

Up to 1 million ad requests a second at peak in Netflix’s framing

Ad servers must be stateless and horizontally scaled, with shared state reaching them through caches they read locally. A per-request write to a shared counter for a popular campaign can become a hot key, so in this design the hot path reads and does not write to shared counters.

Worst-case overspend held to a small single-digit percentage

Spend may be counted from events that arrive late, which avoids a central counter, but the lateness must be bounded, and the campaign stopped early with the margin computed from it. At a 5 minute delay and even pace that margin is about 0.35% of a daily budget; a fast finish multiplies it.

99.9% availability per region, serving no ad rather than failing the page

About 8.8 hours of downtime a year per region, so failover is automatic. More important, the system degrades rather than fails: a timeout or a dead dependency returns no ad, and the feed fills the slot with organic content, so the page goes on.

Spend and impressions reported within 5 minutes

Impression and click events go onto a stream the moment they happen and are aggregated continuously rather than in nightly batches. In this design the same bound is the target for how stale the spend an ad server sees may be, once the pacing cycle and the cache refresh are added in.

Estimates

QuantityWorkingResultWhat it sizes

Requests a day if the peak held all day

100,000 requests a second × 86,400 seconds a day = 8.64 billion

At most about 8.64 billion requests, and so at most that many impressions, a day

An upper bound on impression events and on frequency counters created in one day.

Split of the server time budget

Of the 50 ms floor: retrieval 10 ms, filters 5 ms, ranking 25 ms, auction and response 10 ms = 50 ms; the span up to 70 ms is headroom for the slow tail

Ranking gets about half; the rest is retrieval, filters and the auction

The deadline on each step, and how much headroom is left before the feed gives up on the slot.

Model time per candidate

Taking 300,000 as a working figure for the several hundred thousand active campaigns: 25 ms ÷ 300,000 ≈ 83 nanoseconds each; after retrieval keeps about 300 candidates: 25 ms ÷ 300 ≈ 83 microseconds each

Retrieval buys the model about a thousand times more time per candidate

Why the heavy model runs after retrieval and filters, not on the whole campaign set.

Index memory per server

Assuming about 1 KB per campaign for targeting, bid, budget state and creative ids: 300,000 campaigns × 1 KB ≈ 300 MB

A few hundred megabytes

Holding the complete index in each ad server's memory, which is why no request needs the campaign database.

Overspend from late spend events

Even pace spends a daily budget over 86,400 seconds; spend that arrives 5 minutes (300 seconds) late is 300 ÷ 86,400 ≈ 0.35% of the budget; a fast finish spending at 10 times the even rate makes it about 3.5%

About 0.35% at even pace, a few percent during a fast finish

The margin by which a campaign stops ahead of budget exhaustion, and how the allowed single-digit overspend is met.

Frequency store size as a ceiling

At most 8.64 billion impressions a day; if no user ever saw the same ad twice, that is 8.64 billion counters per daily window; assuming 32 bytes a counter for key hash, count and expiry: 8.64 billion × 32 bytes ≈ 276 GB

Under about 276 GB per daily window, spread over a cluster; repeats share counters so the real figure is lower

The memory of the frequency store, and why counters must expire with their window.

Frequency store traffic

One batched read per ad request: 100,000 reads a second at peak; one increment per served ad: at most 100,000 writes a second

About 100,000 reads and up to 100,000 writes a second at peak

The sharding of the frequency store by user id, and why the read is one round trip, not one per candidate.

Downtime that 99.9% allows

0.1% × 8,760 hours a year ≈ 8.8 hours; 0.1% × 43,200 minutes a month ≈ 43 minutes

About 8.8 hours a year, about 43 minutes a month, per region

Whether failover can involve a person: at this budget it can, but serving must keep returning no ad meanwhile.

Core entities

EntityWhat it holds

Campaign

What an advertiser bought: who to reach, how much to pay and spend, and how often one person may see it.

Ad

One creative under a campaign, with the page a click lands on.

Ad request

One slot asking for one ad, with what is known about the viewer at that moment.

Impression

The record that an ad was served, then seen, and what it cost.

Spend counter

How much of a campaign's budget is gone, as far as the stream has told us.

View counter

How many times one user has seen one ad or campaign inside the current window.

The API

EndpointNotes

GET /v1/ads?slot_id&user_id&context → { ad, impression_id, tracking_urls } or no ad

The hot path. The server answers inside its deadline or returns an empty body, and the feed fills the slot with organic content. The impression id is minted here so the seen event and the auction price can be joined later.

POST /v1/impressions { impression_id, signature } ← sent when the ad is seen

Fired by the client when the ad is on screen. In this design the impression id is signed when it is issued, so the collector can drop forged events, and it keeps the ids it has already accepted for the window so a duplicate is dropped as a replay. Accepted events go onto the stream; nothing on this path is read by the next ad request directly.

POST /v1/campaigns { budget, flight, targeting, bid, frequency_cap } → { campaign_id }

Writes the campaign table. The index builder picks the row up as a delta and pushes it to the ad servers; the serving path itself does not read this table.

GET /v1/campaigns/{id}/delivery → { spend, impressions, pace }

Reads the spend ledger the stream fills, so it reflects spend within the 5 minute reporting bound. Pace is the ratio of spend so far to the target curve at this time of day.

The high-level design

snapshotslotCampaignsAd indexFeedAd serverRanker

Step 1 of 3

The hot path, with the index in memory

Begin with the request that has to be fast. The feed service asks an ad server for one ad for one slot. The server holds the complete campaign set in memory as an index on targeting keys, about 300 MB for 300,000 campaigns at 1 KB each, so matching targeting is a local lookup rather than a database query. Candidates then go to a ranking model that predicts the click rate, and the server picks the winner by bid multiplied by that rate. The index is loaded from the campaign database as a snapshot and kept current with deltas, so the database stays off the hot path.

snapshotslotservedthrottleseencountsCampaignsAd indexFeedAd serverRankerEventsPacerPace cacheCollectorCapsSpend ledgerBilling
The whole design

What breaks first

The deep dives

Picking one ad in time. A user scrolls to a slot and the server has 50 to 70 milliseconds. How does it go from hundreds of thousands of live campaigns to one ad, and what happens when a step runs long?

A concept that helps here: Inverted indexesA concept that helps here: Backpressure and load sheddingA concept that helps here: Change data capture

Budgets and pacing. A campaign has a daily budget and hundreds of servers serve it at once, each seeing spend late. How does it spend evenly through the day and stay inside the overspend tolerance?

A concept that helps here: Stream processing and windowsA concept that helps here: Rate limiting algorithms

Frequency caps. An advertiser allows a user three views of their ad a day. Where do the counts live, how does a request read them in time, and what happens when two requests for one user land at once or in two regions?

A concept that helps here: Cache invalidation and TTLsA concept that helps here: Consistency modelsA concept that helps here: Running in several regions

Concepts this guide uses

Sources

  1. InfoQ: Unpacking how ads ranking works at Pinterest (QCon San Francisco talk by Aayush Mudgal)
  2. Aman Chadha's notes on Pinterest system design, including the Pinterest Engineering post on how ads indexing works
  3. PragmaticLab notes on Agarwal et al., Budget pacing for targeted online advertisements at LinkedIn
  4. arXiv preprint: Smart Fast Finish, preventing overdelivery via daily budget pacing at DoorDash
  5. Google Ads Help: Frequency capping, definition
  6. Redis documentation: INCR
Sign in to report a mistake