Failing open or failing closed
What a system does when a check it depends on (authorization, rate limiting, a policy webhook) can't answer: let the request through or refuse it. The choice sets what that dependency's outage costs, so name it in a design.
What it is
Many requests can't be served until some other component gives a verdict: an authorization service says the caller may do this, a rate limiter says the caller has budget left, an admission webhook says the object is acceptable, a web application firewall says the request looks clean. The failure mode of that check is what happens when the component times out or returns an error instead of a verdict. Failing open treats the missing verdict as permission and lets the request continue. Failing closed treats it as a refusal and rejects the request. Envoy's documentation uses the same two words for its external authorization filter: when the authorization service is unreachable, the request is permitted or denied according to a configuration flag.
The two modes protect different things. Failing open protects the availability of the feature behind the check. Failing closed protects whatever the check guards. Security guidance leans one way: the OWASP page on failing securely says a security mechanism has three outcomes (allow, disallow, exception) and that the exception should take the same path as disallow, so methods like isAuthorized() return false when something throws. Its example is a variable isAdmin initialized to true before a call that can fail: if the call throws, the user is an admin by default. NIST's glossary defines fail secure in the same spirit, as a way of stopping that keeps the system in a secure state even though the failure may still damage something.
| Product and check | When the check can't answer | The switch |
|---|---|---|
Envoy 1.36, HTTP rate limit filter | Lets the request through unless told otherwise (fails open) |
|
Envoy 1.36, external authorization filter | Denies the request (fails closed); |
|
Kubernetes admission webhooks (current docs) |
|
|
AWS Application Load Balancer with AWS WAF |
| Set it to |
The defaults in that table line up with what each check guards. The rate limiter, which shapes traffic, defaults to letting requests through. Authorization, admission policy and the firewall, which keep bad requests out, default to refusing. In each case the vendor also ships the switch, because the right mode depends on what sits behind the check, not on the product.
When to use it
Fail open when the check exists to make things better rather than to stop harm, and a gap without it is tolerable. Rate limits that protect a database from accidental overload, feature flags with a safe default value, personalization or enrichment calls whose absence degrades the response but doesn't make it wrong. Stripe's engineering blog describes its rate limiters this way: exceptions are caught at each layer so that a bug in the limiter or a Redis outage fails open and the API keeps serving. The Kubernetes documentation makes a similar recommendation for mutating webhooks: let them fail open, then check the final state of the object with a validating step. In other words, fail open on the step you can recover from and keep the gate on the step you can't.
Fail closed when a wrong yes is worse than a refused request: authentication, authorization, payment authorization, an admission policy that enforces a security rule, a firewall in front of something attackers probe. Here the guidance is to make the error path and the deny path the same code path, so there is no way for an exception to skip the control. Our advice for both modes: give the call to the check a short timeout, so a slow dependency turns into a quick decision rather than a pile of waiting requests, and treat the timeout as the moment the failure mode applies.
When not to
Don't fail open on a check that decides access or money, even with an availability goal in front of you. The better move is to fail closed and make the dependency itself hard to lose: run several replicas of the authorizer, keep its data local, and give it its own capacity. If you must keep serving through its outage, cache recent allow decisions with a short lifetime and say how long a revoked permission can keep working, because that staleness is the real cost of the cache.
Don't fail closed when the check is advisory and the dependency is less reliable than what it protects: you've tied your uptime to your weakest component. That is the price of the default webhook policy above: rejecting by default can turn webhook downtime into rejected requests that would have passed. Don't treat the choice as binary either. A fallback is often better than both modes: a per-instance local rate limit when the shared counter is unreachable, a last-known policy when the policy service is down, a stale allowlist rather than no allowlist.
Be careful with failing open on a limiter that is your main defense against abuse rather than against accidental overload. During an attack the limiter's dependency may be the first thing to buckle, and failing open removes the defense at that moment. A local limit as the fallback keeps some protection in place.
Compared
| Mode | What users see during the outage | What you risk | Right when |
|---|---|---|---|
Fail open | This check no longer blocks anything; requests proceed without it | The check's protection is absent; an attacker who can slow the dependency gets the same effect | The check shapes traffic or adds value, and a gap without it is cheaper than downtime |
Fail closed | Requests are refused (a 5xx or a deny) | The dependency's downtime is now your downtime; compliant requests get rejected | A wrong yes costs more than a refused request: access, money, security policy |
Fallback (local or cached decision) | Requests succeed under a cruder rule | Stale or coarser decisions for the length of the outage; more code to test | A rough answer is acceptable and you can state the staleness bound |
Split by operation | Reads succeed, sensitive writes are refused | Two code paths to reason about and test | One dependency guards both low-risk and high-risk actions |
In a picture
How to say it in an interview
The common mistake
Quiz
Questions that use this
Related
Sources
- Envoy 1.36.9 documentation: Rate limit (proto), failure_mode_deny
- envoyproxy/ratelimit README: Request deadlines and failure mode
- Envoy 1.36.9 documentation: External Authorization (proto), failure_mode_allow
- Envoy documentation: External Authorization architecture overview
- Kubernetes documentation: Admission Webhook Good Practices
- Elastic Load Balancing documentation: Application Load Balancers (load balancer attributes)
- OWASP: Fail Securely
- NIST CSRC Glossary: fail secure
- Stripe Engineering: Scaling your API with rate limiters