Skip to content

API gateways

A reverse proxy that is the one entry point for clients: it routes each request to the right service and takes on shared work such as TLS, authentication and rate limiting. A strong answer says what stays out of it.

On this page

What it is

An API gateway is a reverse proxy that clients talk to instead of talking to your services. It accepts a request, decides which service should handle it, and forwards it. Along the way it can take on tasks that would otherwise be repeated in each public service: ending the TLS connection, checking credentials, applying rate limits. Some requests it simply passes through to one service; others it fans out to several services and combines the answers.

The alternative is to let clients call each service directly. That works at small scale, but as the number of services grows the client has to know how the system is decomposed, track many endpoints, handle each one's failures, and make several round trips for one screen. Each public service then also has to handle TLS, authentication and throttling itself, and each public endpoint widens the attack surface. A gateway moves those problems to one place.

JobWhat the gateway doesWhat it buys youWhat to watch

Routing

Matches the path, host or header and forwards to one service (layer 7 routing)

Clients see one endpoint; you can move or split services behind it

Routing rules may need updating when services are added or moved

Offloading

Terminates TLS, validates tokens, applies rate limits, logs, caches responses, compresses

Each service skips that work; the client-facing certificate is managed in one place

Each offloaded step adds processing time on the request path

Aggregation

Takes one client call, calls several services, joins the results

Fewer round trips for clients on slow or high-latency networks

Partial failures and timeouts: decide whether to return partial data

What it costs. The gateway is one more hop on the path of each request, and each function it performs there (TLS, token checks, inspection) adds latency. Keeping pooled, keep-alive connections from the gateway to the services recovers part of that cost. Because all traffic passes through it, a gateway with one instance is a single point of failure, and an under-provisioned gateway becomes a bottleneck that slows each service behind it; it needs to be sized for peak bursts rather than average load. Microsoft's guidance also notes that because the gateway terminates client connections and may buffer request bodies, restarting an instance needs connection draining so in-flight requests aren't dropped.

Gateway or load balancer? A load balancer or application delivery controller spreads traffic and may terminate TLS, but it's transparent to the application's logical design: it doesn't change what API the client sees. A gateway does: it can present a client-facing API that differs from the services behind it. Several products do both, so in a design it helps to say which role each box plays. See Load balancing at layer 4 and layer 7 for the traffic-spreading side.

A managed example. Amazon API Gateway throttles at the account level, per region. Its quotas page lists a default of 10,000 requests per second in most regions, with burst capacity from a token bucket holding at most 5,000 requests; in a listed set of regions (Cape Town, Milan, Jakarta and others) the default is 2,500 requests per second with a burst of 1,250. The 10,000 figure can be raised on request, while the burst figure is set by AWS and isn't a quota a customer can change. Numbers like this matter in a design because a managed gateway's own limits become your system's limits unless you ask for more.

When to use it

Reach for a gateway when you have many services (dozens of service types is the scale Microsoft's microservices guidance uses as an example) and the clients are remote: native mobile apps or single-page web apps that would otherwise have to learn your service boundaries and make many trips. Add aggregation when one operation needs several services and the client sits on a high-latency network such as cellular. Use offloading when several services share a concern such as TLS certificates, or when you want network security and throttling owned by a specialized team instead of each service team.

When a web app, a mobile app and third-party integrators need different shapes of API, give each client type its own gateway: the backends for frontends variation. Each one is smaller and changes without touching the others. Our advice: let the team that owns the client own its gateway, so an API change for the phone app doesn't wait on anyone else.

When not to

Skip the gateway when the application is small and the client is a server-side web app calling a few services from inside the same network: direct calls are good enough there, and the hop buys little. Skip aggregation when the client is near the services and latency isn't a concern, or when the real goal is fewer calls to one service across several operations; a batch endpoint on that service does that without a gateway. Skip offloading a concern that is lightweight when the workload is latency-sensitive, because the extra hop can cost more than centralizing saves.

Be wary when the gateway would need service-specific rules that tie its configuration to each service's releases, so a backend change forces a gateway redeploy, or when the gateway team ships more slowly than the service teams and becomes a change bottleneck. And keep business logic out of it: a gateway that aggregates all services and holds their rules turns into a monolithic orchestrator that couples the services it was meant to decouple. One Microsoft architect has called this risk "the new ESB". If composite calls need real logic, put an aggregation service behind the gateway rather than inside it.

Compared

OptionWhat it gives youPick it whenWeaker when

Clients call services directly

No extra hop; each service is public and handles its own TLS and auth

A small system with a server-side web client and a few services

Many services, remote mobile or SPA clients, repeated auth and TLS work

Load balancer or plain reverse proxy

Spreads traffic, terminates TLS, path routing; invisible to the API's shape

You need capacity and TLS, not a different API for clients

Clients need auth, per-client limits or a different API shape

One API gateway

Routing, offloading and optional aggregation in one tier

Many services share concerns and serve one kind of client

It grows into an aggregator of all services; client types diverge

Backends for frontends

A gateway per client type with an API shaped for it

Web, mobile and partners need different data and protocols

One client type; more gateways to run and keep consistent

Aggregation service behind the gateway

Fan-out and joins in a separate service; the gateway stays a thin proxy

Composite calls have different resource needs than routing

Few composite calls; one more service to run

In a picture

Mobile clientAPI gatewayProduct service

Step 1 of 3

Route

The client knows one address. The gateway reads the path and forwards the request to the service that owns it.

How to say it in an interview

The common mistake

Quiz

A phone on a cellular network needs data from three services to show one product page. Which gateway job helps most?
Your one client is your own server-side web app, in the same data center as two backend services. What's the reasonable default?
A client makes many separate calls to one service during one operation. What fits better than gateway aggregation?
Which of these belongs in the gateway rather than in a service?

Questions that use this

Related

Sources

  1. Azure Architecture Center: API gateways
  2. Azure Architecture Center: Gateway Aggregation pattern
  3. Azure Architecture Center: Gateway Offloading pattern
  4. .NET microservices guide: The API gateway pattern versus direct client-to-microservice communication
  5. microservices.io: Pattern: API Gateway / Backends for Frontends
  6. Amazon API Gateway documentation: Amazon API Gateway quotas
Sign in to report a mistake