Skip to content

Failing open or failing closed

What a system does when a check it depends on (authorization, rate limiting, a policy webhook) can't answer: let the request through or refuse it. The choice sets what that dependency's outage costs, so name it in a design.

On this page

What it is

Many requests can't be served until some other component gives a verdict: an authorization service says the caller may do this, a rate limiter says the caller has budget left, an admission webhook says the object is acceptable, a web application firewall says the request looks clean. The failure mode of that check is what happens when the component times out or returns an error instead of a verdict. Failing open treats the missing verdict as permission and lets the request continue. Failing closed treats it as a refusal and rejects the request. Envoy's documentation uses the same two words for its external authorization filter: when the authorization service is unreachable, the request is permitted or denied according to a configuration flag.

The two modes protect different things. Failing open protects the availability of the feature behind the check. Failing closed protects whatever the check guards. Security guidance leans one way: the OWASP page on failing securely says a security mechanism has three outcomes (allow, disallow, exception) and that the exception should take the same path as disallow, so methods like isAuthorized() return false when something throws. Its example is a variable isAdmin initialized to true before a call that can fail: if the call throws, the user is an admin by default. NIST's glossary defines fail secure in the same spirit, as a way of stopping that keeps the system in a secure state even though the failure may still damage something.

Product and checkWhen the check can't answerThe switch

Envoy 1.36, HTTP rate limit filter

Lets the request through unless told otherwise (fails open)

failure_mode_deny: true refuses traffic when the rate limit service doesn't respond

Envoy 1.36, external authorization filter

Denies the request (fails closed); failure_mode_allow is false unless set

failure_mode_allow: true accepts the request even if the authorizer is unreachable or returns a 5xx

Kubernetes admission webhooks (current docs)

failurePolicy defaults to Fail: the API server rejects the request

failurePolicy: Ignore lets the request proceed and skips the webhook

AWS Application Load Balancer with AWS WAF

waf.fail_open.enabled is false by default: requests aren't routed to targets when the balancer can't reach WAF

Set it to true to route requests anyway

The defaults in that table line up with what each check guards. The rate limiter, which shapes traffic, defaults to letting requests through. Authorization, admission policy and the firewall, which keep bad requests out, default to refusing. In each case the vendor also ships the switch, because the right mode depends on what sits behind the check, not on the product.

When to use it

Fail open when the check exists to make things better rather than to stop harm, and a gap without it is tolerable. Rate limits that protect a database from accidental overload, feature flags with a safe default value, personalization or enrichment calls whose absence degrades the response but doesn't make it wrong. Stripe's engineering blog describes its rate limiters this way: exceptions are caught at each layer so that a bug in the limiter or a Redis outage fails open and the API keeps serving. The Kubernetes documentation makes a similar recommendation for mutating webhooks: let them fail open, then check the final state of the object with a validating step. In other words, fail open on the step you can recover from and keep the gate on the step you can't.

Fail closed when a wrong yes is worse than a refused request: authentication, authorization, payment authorization, an admission policy that enforces a security rule, a firewall in front of something attackers probe. Here the guidance is to make the error path and the deny path the same code path, so there is no way for an exception to skip the control. Our advice for both modes: give the call to the check a short timeout, so a slow dependency turns into a quick decision rather than a pile of waiting requests, and treat the timeout as the moment the failure mode applies.

When not to

Don't fail open on a check that decides access or money, even with an availability goal in front of you. The better move is to fail closed and make the dependency itself hard to lose: run several replicas of the authorizer, keep its data local, and give it its own capacity. If you must keep serving through its outage, cache recent allow decisions with a short lifetime and say how long a revoked permission can keep working, because that staleness is the real cost of the cache.

Don't fail closed when the check is advisory and the dependency is less reliable than what it protects: you've tied your uptime to your weakest component. That is the price of the default webhook policy above: rejecting by default can turn webhook downtime into rejected requests that would have passed. Don't treat the choice as binary either. A fallback is often better than both modes: a per-instance local rate limit when the shared counter is unreachable, a last-known policy when the policy service is down, a stale allowlist rather than no allowlist.

Be careful with failing open on a limiter that is your main defense against abuse rather than against accidental overload. During an attack the limiter's dependency may be the first thing to buckle, and failing open removes the defense at that moment. A local limit as the fallback keeps some protection in place.

Compared

ModeWhat users see during the outageWhat you riskRight when

Fail open

This check no longer blocks anything; requests proceed without it

The check's protection is absent; an attacker who can slow the dependency gets the same effect

The check shapes traffic or adds value, and a gap without it is cheaper than downtime

Fail closed

Requests are refused (a 5xx or a deny)

The dependency's downtime is now your downtime; compliant requests get rejected

A wrong yes costs more than a refused request: access, money, security policy

Fallback (local or cached decision)

Requests succeed under a cruder rule

Stale or coarser decisions for the length of the outage; more code to test

A rough answer is acceptable and you can state the staleness bound

Split by operation

Reads succeed, sensitive writes are refused

Two code paths to reason about and test

One dependency guards both low-risk and high-risk actions

In a picture

answerederroropenclosedRequestAsk the checkApply its verdictFailure mode?Let it throughRefuse itCount and alert
A missing verdict is a decision too: the configured failure mode picks the path, and a fallback that fires silently hides the outage.

How to say it in an interview

The common mistake

Quiz

Your API checks a per-customer rate limit in Redis before each request. Redis becomes unreachable. Which behavior fits best?
An admission webhook enforces that no container runs as root. Its pods are restarting and the API server can't reach it. With the Kubernetes default failure policy, what happens to a Pod create?
Which of these checks should fail closed?
A service fails open when its fraud-scoring dependency is down, and nothing records it. What is the main problem?

Questions that use this

Related

Sources

  1. Envoy 1.36.9 documentation: Rate limit (proto), failure_mode_deny
  2. envoyproxy/ratelimit README: Request deadlines and failure mode
  3. Envoy 1.36.9 documentation: External Authorization (proto), failure_mode_allow
  4. Envoy documentation: External Authorization architecture overview
  5. Kubernetes documentation: Admission Webhook Good Practices
  6. Elastic Load Balancing documentation: Application Load Balancers (load balancer attributes)
  7. OWASP: Fail Securely
  8. NIST CSRC Glossary: fail secure
  9. Stripe Engineering: Scaling your API with rate limiters
Sign in to report a mistake