Flow Control & Backpressure

In a synchronous system, if a downstream service is slow, it blocks the upstream thread. In an asynchronous system, messages just keep piling up in the queue until the disk is full or the consumer crashes with an OutOfMemory error.

This is why Flow Control is the most critical operational component of a high-volume system. You must design your system to say “No” before it breaks.

1. Backpressure vs. Load Shedding

Backpressure (The Signal)

The downstream service signals the upstream to slow down.

  • Analogy: A bottleneck on a highway that causes traffic to back up miles behind it.
  • Mechanism: TCP windows, Reactive Streams pull-based demand.

Load Shedding (The Rejection)

The system proactively drops requests it knows it cannot process.

  • Analogy: A nightclub with a “One In, One Out” policy.
  • Mechanism: Priority queues, TTL (Time-to-Live), and token buckets.

2. Interactive Flow Simulator

See how different strategies impact system stability during a traffic spike.

Producer

Rate: 10/s

Queue

0 messages

Consumer

Capacity: 5/s
Select a protection mode to start...

3. Reactive Streams & CDC

Reactive Streams (The Standard)

A programming standard for asynchronous stream processing with non-blocking backpressure.

  • The Contract: Consumer says “I can handle 10 more.” Producer sends exactly 10.
  • Implementations: Java’s Project Reactor, RxJS, Akka Streams.

Staff-Level Insight: The CDC Schema Trap

When using CDC (e.g., Debezium), your “flow” is at the mercy of the database schema.

  • The Failure: A developer adds a NOT NULL column to the source DB.
  • The Impact: If your downstream consumer (e.g., a Go app) hasn’t been updated to handle the new field, it will fail to deserialize the message.
  • The Staff Move: Always use Schema Evolution tools (like Confluent Schema Registry). If a breaking change occurs, the CDC pipe should crash immediately rather than silently dropping data or filling logs with errors.

CDC (The Ultimate Backpressure)

Change Data Capture (CDC) using systems like Debezium provides a natural flow control.

  1. Log-based: Debezium reads the DB Transaction Log.

  2. Implicit Delay: If the consumer is slow, the “pointer” in the log just moves slowly. The database is never overwhelmed by the integration flow.

4. Operational Hazard: Buffering Bloat

Many engineers think they have backpressure because they use an async queue. In reality, they have Buffering Bloat.

The Pathology: Delayed OOM

If your Go channels or Java BlockingQueue are unbounded (or set to 10k+), they will absorb 100% of a traffic spike.

  • The Symptom: Latency spikes to 30 seconds as messages sit in the queue.
  • The Crash: Eventually, the JVM or Go runtime runs out of memory (OOM) and the service dies before it ever sends a “stop” signal to the upstream.
  • Staff Rule: Every queue must have a Hard Limit and a Rejection Policy (Drop Oldest, Drop Newest, or Shed Load).

5. Staff Insight: The “Livelock” of Load Shedding

Load shedding is meant to protect the system, but it can actually kill it.

The Livelock Scenario

If you reject requests with a 503 Service Unavailable, you still have to:

  1. Terminate the TLS connection.
  2. Parse the HTTP Headers.
  3. Run the Auth middleware (to prevent DDoS of the sytem).
  4. Generate and send the 503 response.

If the “Cost of Rejection” is 1ms of CPU, and the “Cost of Processing” is 2ms, then a massive spike will still consume 100% of your CPU just rejecting traffic. The Solution: Move load shedding as far to the Edge (Load Balancer/API Gateway) as possible to protect your core compute.

4. Staff Math: Safety & Bufferbloat

A queue is not a solution; it’s a delay injector.

4.1. Little’s Law for Bottlenecks

How deep can your queue be before you violate your SLO? [ \textbf{L} = \lambda \times \textbf{W} ]

  • Example: Your SLO is 500ms ($W = 0.5$). Your service handles 200 RPS ($\lambda$).
    • Max Concurrency (L): $200 \times 0.5 = \mathbf{100 \text{ requests}}$.
    • If your service has 50 threads, your Max Queue Size must be $\mathbf{50}$. If the queue hits 51, you are already violating your 500ms SLO.

4.2. The Bufferbloat “Latency Tax”

If you have a 10,000 deep queue in front of a service that only processes 100 RPS: [ \text{Queuing Delay} = \frac{\text{Queue Depth}}{\text{Processing Rate}} \approx \frac{10,000}{100} = \mathbf{100 \text{ seconds}}. ]

  • Impact: Any user request that enters this queue is effectively dead. By the time it’s processed, the user has already closed their browser.
  • The Staff Move: Use LIFO (Last-In-First-Out) Queues during overload. The “freshest” requests get processed first, giving some users a good experience, while the “oldest” ones are eventually dropped.

4.3. The Cost of Rejection (Admission Control)

Sending a 503 is cheaper than a 200, but it’s not free.

  • Math: If a 200 OK takes 5ms of CPU and a 503 Service Unavailable takes 0.5ms (for TLS handshake + logging):
  • Limit: If your CPU is at 100%, you can process 200 reqs/s. If you switch to rejecting everything, you can only handle 2,000 reqs/s of pure rejection before the server crashes anyway.
  • Defense: This is why you must perform Edge-Level Load Shedding (at the Load Balancer or CDN), where the rejection cost is offloaded from your application.

Staff Takeaway

A system without flow control is a time bomb.

  • **Backprexcerpt: “Managing complex distributed state across microservices. Comparing Choreography vs Orchestration and implementing Sagas for absolute consistency.” image: /Users/laxmansharma/.gemini/antigravity/brain/f6c3a85a-4b26-4d7d-8c43-cded1a3fe336/async_module_hero_saga_1768145867737.png —

Saga Pattern

When you split a monolith into microservices, you lose ACID transactions across the whole system. at the cost of some events.

  • Staff tip: Always set a max-retention-time and max-size-gigabytes on your Kafka topics to prevent a slow consumer from taking down the entire broker cluster’s disk.