YOUR BRIEF
What to work through
Work through an A/B testing platform as a spoken system design interview. Start by clarifying the scope, sketch a working design and explain the trade-offs behind your choices.
- Create an experiment: variants, traffic split, targeting and metrics
- Give each user a variant that stays the same across sessions and devices
- Log an exposure when a user actually gets the variant
- Compute each metric per variant, with confidence intervals
- Ramp traffic up, and alert or pause when a guardrail metric gets worse
BEFORE YOU START
Constraints to discuss
- Assignment fits inside callers’ latency budgets: LinkedIn evaluates in 700 ns on average, 3 μs at p99
- Thousands of experiments at once: over 1,000 at Uber, about 35,000 at LinkedIn
- Assignment keeps working when the platform’s backend is down
- Results stay trustworthy: split checks, and no peeking without a sequential test
Need a starting point?
Watch for these common gaps as you prepare:
- Random assignment on each request, so users flip between variants
- Counting everyone assigned instead of everyone exposed, which dilutes the effect
- Peeking at results every day with a fixed-horizon test
- No sample ratio check, so a broken split goes unnoticed
- A remote call for every assignment in the product’s hot path
- One hash for every experiment, so the same users always land in treatment
Why this question is in the collection
These sources connect the problem to company interview reports or preparation guides. They do not guarantee what a company will ask you.
- Affirm: PracHub lists two Affirm reports of an A/B testing platform: an onsite from February 2026 and a technical screen from March 2026.
- Airbnb: PracHub lists an Airbnb onsite from April 2026: an A/B testing platform with stable assignment, exposure and outcome events, and gradual ramps.
- Microsoft: PracHub lists a senior Microsoft hiring-manager screen from June 2026: a deep dive on an experimentation platform’s validations, latency budgets and health checks.
- Amazon: Aced’s Amazon guide, drawn from the interviewers and candidates it spoke with, lists an A/B experimentation platform as a Bar Raiser’s favorite.