Caching Strategies Hero

Why Cache?

In a distributed system, a database read might take 10ms-50ms. A cache read (like Redis) usually takes <1ms. When you’re serving 100k requests per second, that delta is the difference between a snappy app and a complete system meltdown.

However, caching isn’t just about speed. It’s about database protection.

Interactive Strategy Simulator

Select a strategy and click Send Request to see the data flow.

App
Cache (Redis)
Database (SQL)
Waiting for input...

1. Topologies: Where does the cache live?

Beyond when to cache (strategies), you must decide where to cache.

A. In-Memory (Local)

The cache lives in the app process heap (e.g., Guava, Caffeine).

  • Speed: Nanoseconds.
  • Cons:
    • Inconsistency: Server A has v1, Server B has v2.
    • Memory Pressure: Consumes app RAM.

B. Distributed (Remote)

A centralized cluster (Redis/Memcached) accessed over the network.

  • Speed: Milliseconds (Network round-trip).
  • Pros: Consistent view for all servers. Scales independently.

C. Sidecar / Mesh

A proxy (Envoy, Mcrouter) sits next to the app container.

  • Role: Handles routing, sharding, and failover transparently. The app just says get(key) to localhost.

2. Partitioned vs. Replicated

When your cache grows beyond one node, how do you split data?

Partitioned (Sharding)

Consistent Hashing splits the keyspace. Key A -> Node 1, Key B -> Node 2.

  • Pros: Infinite storage scale.
  • Cons: Hot Keys. If Key A is Twitter’s homepage, Node 1 melts while Node 2 sleeps.

Replicated

Every node has a copy of all hot keys.

  • Pros: High read throughput for hot keys.
  • Cons: Expensive updates (must update all replicas).

[!TIP] Hybrid Approach: Use Partitioning for the long tail of data, and Local Caching (L1) or Replication for the top 1% hot keys.

3. Caching Patterns

Cache-Aside (Lazy Loading)

This is the most common strategy. The App is responsible for everything.

  • Read: App checks cache. On miss, it reads from DB and updates cache.
  • Write: App writes to DB, then invalidates (deletes) the cache entry.

[!IMPORTANT] Always delete the cache on write, don’t update it. This prevents race conditions where an old write overwrites a newer one. (See the Coordination chapter for why even this isn’t enough).

Write-Through

The Cache sits in front of the DB. The app only talks to the cache.

  • Write: App writes to cache. Cache synchronously updates the DB.
  • Pros: Data is never stale.
  • Cons: Higher write latency because you’re waiting on two systems.