API Gateway Pattern

In 2012, Netflix faced a problem: they supported 800+ device types, each needing different data shapes from the same backend services. Their single REST API was a mess of conditional logic and over-fetching. Their solution — the Backend for Frontend (BFF) pattern and their gateway Zuul — became the architectural blueprint for every major API gateway used today. By 2023, Kong’s API gateway was handling over 1.5 trillion API calls per year across its users.

The API Gateway is the “front door” of your system. Get it wrong and you have a single point of failure that takes down your entire platform. Get it right and it becomes your most powerful cross-cutting concern platform — handling auth, rate limiting, routing, and observability in one place.

[!IMPORTANT] In this lesson, you will master:

  1. Cross-Cutting Concerns: How offloading Auth, Throttling, and SSL from business logic services improves both security and scalability.
  2. The BFF Pattern: Why Netflix-style device-specific gateways beat “One-Size-Fits-All” APIs.
  3. Physical Overhead: Managing TLS handshakes and Hop Latency in a layered architecture.

No. You introduce a “Front Door”: the API Gateway.

Hardware-First Intuition: The Gateway handles SSL/TLS Termination. The complex math of RSA/ECC handshakes happens here. Modern Gateways use AES-NI instructions (hardware acceleration) to handle 100k+ handshakes per second. By terminating SSL at the edge, your internal microservices can talk “Naked HTTP”, which saves massive CPU cycles across your entire cluster.

1.0 Edge Computing & Global Steering (Anycast)

Modern API Gateways often live at the Edge (the CDN layer). Using Anycast BGP routing, a request to api.example.com is automatically routed to the Gateway node physically closest to the user (e.g., Tokyo for a user in Japan, London for someone in the UK).

This moves routing logic even further out, allowing you to run small pieces of business logic (like Auth checks or A/B testing headers) in Edge Workers (Cloudflare Workers, AWS Lambda@Edge) before the request ever hits your main data center.

1. Core Responsibilities

[!NOTE] War Story: The “Thundering Herd” of 2018 A popular ticketing platform experienced a massive spike in traffic when a highly anticipated concert went on sale. The API Gateway was not configured with a global rate limit, relying only on local limits. The result? A “Thundering Herd” of requests overwhelmed the backend services, causing a cascading failure. The platform crashed for hours. The fix? Implementing global rate limiting via Redis at the Gateway layer, ensuring precise control over traffic quotas and protecting the fragile backend.

1.1 Request Routing

The gateway acts as a Reverse Proxy, routing requests to the appropriate microservice.

  • GET /users → User Service (10.0.0.1)
  • POST /checkout → Payment Service (10.0.0.5)

Interactive Visualizer: Routing Logic Simulator

Define simple path-based rules and test them. Type a path like /api/users/123.

Routing Rules (Config)
/api/users* → User Service
/api/orders* → Order Service
/api/pay* → Payment Service
* → 404 Not Found
Test Request
Waiting...
API Gateway
User Svc
Order Svc
Pay Svc

1.2 Authentication & Authorization (AuthN/AuthZ)

Instead of implementing JWT validation in every single microservice (DRY violation), you do it once at the Gateway.

  • The Gateway validates the token.
  • It passes the request to the backend with a header: X-User-ID: 123.

1.3 Rate Limiting

The process of controlling the rate of traffic sent or received by a network interface controller or used by an application.

Staff Engineer Tip: Choose between Local and Global Rate Limiting.

  • Local: Each Gateway node keeps its own counter in RAM. Fast but “Approximate”.
  • Global: All Gateway nodes check a centralized Redis store. Accurate but adds Network Hop Latency.

Common Algorithms:

  1. Token Bucket: Allows bursts. (Tokens refill at rate R).
  2. Leaky Bucket: Smooths out traffic (constant outflow).
  3. Fixed Window: “100 reqs per minute”. Can suffer from spikes at window edges.

1.4 Circuit Breaking

If the “Payment Service” is down, the Gateway should stop sending requests to it immediately to prevent cascading failures. It “Trips the Circuit” and returns 503 Service Unavailable instantly.

Interactive Visualizer: Rate Limit & Circuit Breaker Arena

  • Token Bucket: Watch tokens refill. Bursts are allowed.
  • Circuit Breaker: Simulate a “Service Failure”. Watch the Gateway stop forwarding requests (Open State) and then try to recover (Half-Open).
Token Bucket
Available: 5
> System initialized. Mode: Token Bucket.

2. Advanced Patterns

2.1 Backend for Frontend (BFF)

A Mobile App has different needs (small screen, less data) than a Desktop Web App. Instead of one giant “One Size Fits All” API, create separate Gateways:

  • Mobile Gateway: Strips down responses, aggregates data to save battery.
  • Web Gateway: Returns full rich data. This is the BFF pattern.

BFF Architecture Diagram

Backend for Frontend (BFF) Pattern
Web App
Mobile App
Web BFF
Aggregates Full Data
Mobile BFF
Minifies JSON
Shared Microservices
User Service
Payment Service
Catalog Service

2.2 Case Study: Netflix API Gateway (The Birth of BFF)

[!NOTE] Netflix’s BFF pattern with Zuul is documented in their tech blog. The key insight was that device-specific aggregation at the edge eliminated chatty microservice patterns on mobile, reducing round trips by 10x.

2.2.1 The Scenario

Netflix runs on thousands of device types: Smart TVs, iPhones, Androids, PlayStations, Xboxes. Each device has different screen sizes, memory limits, and bandwidth constraints.

2.2.2 The Challenge: The One-Size-Fits-None API

A “Generic” REST API failed them.

  • TV: Wants high-res artwork, 4K metadata, and 50 rows of content.
  • Mobile: Wants low-res thumbnails, minimal metadata, and only 5 rows (to save battery/data).
  • Result: The mobile app was fetching megabytes of unused data (Over-fetching) or making 20 different calls to get what it needed (Under-fetching/Chattiness).

2.2.3 The Solution: Backend for Frontend (BFF) with Zuul

Netflix introduced the BFF Pattern using their gateway, Zuul. Instead of one API, they built a “Adapter” for each device type.

The “Aggregation” Flow

  1. Client Request: The PlayStation app sends ONE request: GET /playstation/home.
  2. Fan-out: The Gateway (BFF) calls 5+ microservices in parallel:
    • Recommendation Service
    • Continue Watching Service
    • Billing Service (Is account active?)
  3. Stitching: The Gateway combines these responses into a single JSON tailored exactly for the PlayStation UI.
  4. Response: The client gets one perfect payload.

2.2.4 Result

  • Latency: Reduced by 50% (fewer round trips).
  • Developer Velocity: The UI teams could change their data requirements by tweaking their BFF adapter without waiting for the backend teams to change the core microservices.

3. Observability: The RED Method

Since all traffic flows through the Gateway, it is the perfect place to measure system health. We use the RED Method:

  1. Rate: Number of requests per second.
  2. Errors: Number of failed requests per second.
  3. Duration: How long each request takes (Latency).

Memory hook: RED = “Are we Red?” — if Rate drops OR Errors spike OR Duration increases, you have an incident.

[!TIP] The RED method is complementary to the USE method (Utilization, Saturation, Errors) for infrastructure metrics. Use RED at the Gateway (application layer) and USE for CPU/memory/disk (system layer).

4. API Gateway vs Service Mesh

A common point of confusion. When do you use which?

Feature API Gateway Service Mesh (e.g., Istio)
Location Edge (Entry point) Internal (Sidecar proxy)
Traffic Type North-South (Client → Server) East-West (Service → Service)
Focus Auth, Rate Limiting, Composition Retries, mTLS, Circuit Breaking
User External Clients Internal Services

Deep Dive: Sidecar Pattern

In a Service Mesh, every microservice has a tiny “Sidecar” container running next to it (like Envoy).

  • Service A doesn’t talk to Service B directly.
  • Service A talks to Sidecar A.
  • Sidecar A talks to Sidecar B.
  • Sidecar B talks to Service B.

This allows you to upgrade encryption (mTLS) or add retry logic without changing a single line of application code.

4.2 Observability Headers: The RED Method in Practice

To implement the RED Method, Gateways often inject observability headers into responses, allowing monitoring tools like Prometheus or Databdog to aggregate them:

X-Request-Duration: 42ms
X-Gateway-Node: eu-west-1a
X-Rate-Limit-Remaining: 4999

Best Practice: Use Both. The Gateway handles external traffic, and the Mesh handles internal communication.

5. Summary

  • API Gateway is the single entry point for all clients — it handles Auth, Rate Limiting, SSL, and Routing.
  • Use the BFF Pattern to tailor APIs for different clients (Mobile vs Web).
  • Service Mesh handles internal (East-West) traffic; Gateway handles external (North-South).
  • Popular tools: Kong, Zuul, AWS API Gateway, nginx.
  • RED Method: Rate, Errors, Duration — the three signals at your Gateway that define system health.

Staff Engineer Tip: Local vs Global Rate Limiting. This trips most senior engineers. Local rate limiting (per-node counters in RAM) is fast but approximate — 3 Gateway nodes × 100 req/min means a single user can actually make 300 req/min. Global rate limiting (shared Redis atomic counters) is accurate but adds ~0.5ms network hop. The right answer: use local rate limiting for DDoS protection (speed > precision) and global rate limiting for per-user API quotas (precision > speed). Document which algorithm you’re using in your runbook.

You have completed Module 03! Review the key concepts in the Module Review before moving on.