Skip to content

How to design real-time fraud detection

The first sections are free. The deep dives, level expectations and quiz are part of Pro.

On this page
Practice this question

The problem

A real-time fraud check sits on the payment path: before a card payment is sent for authorization, it gets a decision to approve, block or send to review, and the payment waits for that decision. The answer turns on a hard deadline of under 100 ms that the check has to meet at about 2,300 payments a second, which argues against any lookup whose cost grows with a card's history. Counts for the card, the account and the device are kept ready in a key-value store, read in parallel, scored by a model, then passed through rules that analysts can change without a deploy. The second half of the design is the learning loop: the truth about a payment arrives as a chargeback 30 to 90 days later, so the system records, for each decision, the feature values behind it, and labels are joined back to that record.

Clarifying questions

AskWhy it mattersA typical answer

Where in the payment flow does the check run, and how long may it take?

A check before authorization has a hard deadline and blocks the customer's purchase; a check after authorization loses that deadline but has to reverse the payments it later judges fraudulent. The two are different systems.

Before authorization, on the payment path, with a budget of under 100 ms for the whole decision.

How many payments at peak, and how rare is fraud?

The peak rate decides how many lookups a decision can afford, and the fraud rate decides how to judge the model and how much manual review a given block threshold produces.

About 2,300 payments a second at peak, about 1,350 on average over the busiest weekend, and roughly one payment per thousand fraudulent.

What are the possible outcomes, and who acts on a review?

Three outcomes instead of two add a manual review queue, an analyst tool and a label source; they also give you a softer action than block when a rule is new.

Approve, block or review. Flagged payments go to a queue that analysts clear, and their verdicts come back as labels.

When do we learn whether a decision was right?

Label delay shapes the whole training loop: how long decision logs are kept, how often retraining is worth doing, and whether earlier signals exist.

Chargebacks arrive 30 to 90 days after the payment. Card-network fraud reports can arrive a few days earlier than the chargeback itself.

Who changes the rules, and how fast do they need to land?

If analysts ship rules, the rules are data with their own store, editor and rollout controls; if engineers do, they are code and wait for a deploy.

Analysts add and change rules without a deploy. Fraud tactics shift often, so a new rule should be live within minutes, in a mode that can be checked first.

What happens to a payment when the check itself fails?

It fixes the fallback: whether a slow or dead risk service lets payments through, blocks them, or decides by a cheaper path, and whether that can differ per merchant.

The payment still gets a decision. Which decision is yours to argue for: let payments through, or stop them, for as long as the check is unavailable. Pick a side and say why.

The requirements menu

RequirementScopeWhy

Score every payment and return approve, block or review before authorization

Core

The product. It puts the check on the payment path, which is where the latency budget comes from.

Combine fixed rules (block lists, velocity limits) with a model score

Core

Rules catch a known pattern the same day; the model catches what nobody wrote a rule for. The design needs both and an order between them.

Let analysts add or change rules without a deploy

Core

Fraud tactics shift in days. Rules stored as data and refreshed into memory make this possible; rules as code do not.

Send flagged payments to a manual review queue

Core

A third outcome between approve and block. It needs a queue, an analyst tool and a path for verdicts to come back as labels.

Learn from chargebacks and fraud reports that arrive weeks later

Core

The feedback loop. It forces a decision log with feature values, a label pipeline and a retraining cadence that respects the delay.

Ask the customer for a step-up check (3DS) as a fourth outcome

Optional

Worth naming: it turns some would-be blocks into a challenge the real cardholder can pass. Leave it unless asked, because the challenge flow lives in the checkout, not in the risk service.

Let merchants write their own rules and set their own thresholds

Optional

Real products offer it, and it adds a per-merchant dimension to the rules store and to the alerts. Take it if the interviewer leans toward the product; skip it in a pure infrastructure round.

Show merchants why a payment was blocked

Optional

The interviewer may raise it late. It costs little if each logged decision already carries which rules fired and which features pushed the score; it costs a redesign if the log holds just a score.

Per-merchant allow lists for trusted customers

Optional

Cheap to add as a rule type that overrides the others, and it lowers false positives for repeat buyers. Mention it; do not spend board time on it.

Training the model and choosing its algorithm

Out of scope

Say what the model consumes and produces (features in, a score out) and how it is judged, then leave the algorithm to the ML round.

Account takeover and login risk

Out of scope

A different checkpoint with different features (sessions, password resets). The same engine can host it later, which is worth one sentence.

The card network’s and the issuing bank’s own checks

Out of scope

They run after your decision and you do not control them. Their fraud reports do matter to you as labels, which is where they re-enter the design.

Handling disputes and refunds

Out of scope

An operations workflow on top of the payments ledger. You consume its outcome as a label and stop there.

Non-functional requirements

RequirementWhat it forces

A decision in under 100 ms, as Stripe Radar does

Nothing on the hot path may grow with a card's history: counts are precomputed and read by key, lookups run in parallel so latency is one round trip rather than a sum, and the call from the payment service into the risk check carries a timeout inside that budget.

Fraud is rare: about 1 payment in 1,000 at Stripe

Accuracy is a useless yardstick (approving everything scores about 99.9%), so the model is measured by precision and by recall, training samples the good payments down, and a review queue sized for the rare positives is realistic.

Few good payments blocked: Radar wrongly blocks about 0.1% of legitimate payments

Block is the expensive outcome, so a new rule or model runs in shadow before it acts, review rather than block is the softer action for uncertain cases when the analysts have capacity, and the block rate of each merchant and of each rule is alerted on so a bad rule can be caught early and switched off.

A payment still gets a decision when the fraud check is slow or down

A timeout on the call and a defined fallback behind it: the rules by themselves, or a default chosen for each merchant. Slow actions such as bans and opening reviews leave the request through a queue so they can fail without taking the payment with them.

Estimates

QuantityWorkingResultWhat it sizes

Peak payment rate

137,000 transactions a minute at the 2024 Black Friday to Cyber Monday peak ÷ 60 seconds ≈ 2,283 a second

About 2,300 payments a second

The load each component on the payment path sees at the worst minute: risk service, feature store reads and model calls.

Average rate and peak-to-average ratio

465 million transactions over the four-day weekend ÷ 345,600 seconds ≈ 1,345 a second; 2,300 ÷ 1,350 ≈ 1.7

About 1,350 a second on average; the peak is about 1.7 times the average

How much headroom to provision: the peak is not far above the average, so the design is sized for the peak and runs near it for days.

Feature store reads a second at peak

One read per entity (card, account, device, IP) = 4 reads a decision; 4 × 2,300 ≈ 9,200 reads a second

About 9,200 key-value reads a second, issued in parallel so each decision waits one round trip

The read load on the feature store, which the design spreads across partitions and replicas; set it against a history scan per payment on the main database, whose cost grows with each card's history.

A latency budget inside 100 ms

An example split, assumed for this design rather than measured: payment service timeout on the risk call 80 ms, leaving 20 ms of the 100 for its own work. Inside the risk service, assume parallel feature reads 15 ms, model scoring 20 ms, rules 30 ms (300 rules at about 30 ms) = 65 ms, leaving 15 ms for network hops and serialization

A plan that fits, with the rules engine and the model as the two largest fixed costs

Whether the serial chain (features, then model, then rules) fits the budget, and which step to look at first when it does not.

Decision log size

116 million decisions a day; if the over 1,000 features are stored at about 4 bytes each, a row is about 4 KB; 116 million × 4 KB ≈ 464 GB a day; labels take up to 90 days, so 90 × 464 GB ≈ 42 TB must be kept before the last labels for a day's decisions arrive

About 464 GB a day, about 42 TB held until labels are complete

The decision log: a columnar store in object storage, not a row database, and a retention policy tied to the label delay.

Fraud and false positives a day

116 million a day ÷ 1,000 ≈ 116,000 fraudulent payments; 0.1% of about 116 million good payments ≈ 116,000 good payments wrongly blocked

About 116,000 fraudulent payments a day, and roughly as many good payments blocked by mistake

The cost of each side of the threshold: at this false-positive rate the blocked good payments are about as many as the fraud, so moving the threshold is a business decision, not a tuning detail.

Accuracy of a model that approves everything

Fraud is 1 in 1,000, so approving all payments is right 999 times in 1,000 = 99.9% accurate while catching no fraud

99.9% accuracy with zero fraud caught

Why the model is measured by precision and by recall at a chosen threshold instead of accuracy.

Age of the newest fully labeled payments at training time

Chargebacks arrive 30 to 90 days after the payment, so a model trained today has complete chargeback labels for payments at least 90 days old, while labels for the last 30 days are still arriving

Training data lags the present by one to three months

The retraining cadence: hourly retraining brings almost no new labels, so weekly or monthly is the sensible range, with rules covering the gap.

Core entities

EntityWhat it holds

Payment

The request being judged: who is paying whom, with what, from where.

Feature

One precomputed value about one entity, such as payments on this card in the last hour.

Rule

An analyst's predicate over features and the score, with the action it takes and the state it is in.

Decision

What the system said about one payment and the exact inputs it said it from.

Label

A later verdict about a payment: a chargeback, a network fraud report or an analyst's call.

The API

EndpointNotes

POST /v1/risk/score { payment_id, card, account, merchant, amount, device, ip } → { action, score, reasons }

The hot path, called by the payment service with a timeout inside the 100 ms budget. Idempotent on payment_id, so a retry after a timeout returns the same decision instead of scoring twice. reasons carries which rules fired and the top features, which is what the merchant-facing explanation is built from later.

POST /v1/rules { checkpoint, predicates, action, status } → { rule_id }

In this design a new rule is saved with status shadow, and the analyst switches it to active with a later update once its logged results look right. The service validates the predicate syntax and can replay it over recent decisions to show what it would have matched. Rules are data: nothing here triggers a deploy.

POST /v1/labels { payment_id, source, is_fraud } ← chargeback and review feeds

Fed by the label pipeline, not by humans directly. Deduplicated on the external reference, because the same fraud report can arrive from the network and the processor. Appends to the label table that training and rule evaluation join to the decision log.

GET /v1/reviews?status=open → flagged payments for analysts

Backed by the review queue. In this design the list is ordered by amount and age, so large payments and cases close to expiring tend to be looked at first, and recording a verdict closes the case and posts a label with source review, so manual decisions feed training too.

The high-level design

timeoutdecisionPayment serviceRisk serviceRules engineDecision log

Step 1 of 4

A risk service on the payment path, with a timeout

Start with the call itself. The payment service, before it sends the payment to the card network, calls a risk service and waits for approve, block or review. The call carries a timeout well inside the 100 ms budget, so a slow check can't hold the payment hostage. The risk service runs the rules (block lists, velocity limits) and writes each decision to a log. This already settles the first design argument: the check is synchronous and has a deadline, so whatever we add later has to fit inside it.

timeoutcountsparallelrefreshdecisionactionsjoinmodelPayment serviceRisk servicePayment eventsStream processorFeature storeModel scoringRules engineAnalyst toolRules storeDecision logAction queueReview queueChargebacks,reportsLabel pipelineTraining job
The whole design

What breaks first

The deep dives

Scoring inside the latency budget. A payment arrives and the check has under 100 ms to say approve, block or review. What runs, in what order, how long does each step take at about 2,300 payments a second, and what does the payment path do when the check is slow or down?

A concept that helps here: Fan-out on write and on readA concept that helps here: Failing open or failing closedA concept that helps here: Walking one request through the design

Fresh features and late labels. A stolen card is used ten times in one minute, and the chargeback for its first use arrives two months later. How does a count like payments per card in the last minute stay current, and how does a label that arrives in two months reach the model?

A concept that helps here: Stream processing and windowsA concept that helps here: Approximate counting

Changing rules and models safely. Fraudsters change tactics this week and an analyst needs a new rule live today, while a rule that misfires meets about 2,300 payments a second. How does a rule go from the analyst to live traffic, how is a new model shipped the same way, and how is a bad one caught and stopped?

A concept that helps here: Stating a trade-off

Concepts this guide uses

Sources

  1. Stripe: How we built it: Stripe Radar
  2. Uber Engineering: Mastermind, using Uber Engineering to combat fraud in real time
  3. Uber Engineering: How Uber processes early chargeback signals
  4. Stripe newsroom: Black Friday through Cyber Monday 2024 results
  5. Stripe documentation: Fraud prevention rules
Sign in to report a mistake