A Junior Engineer thinks of a CDN as “S3 for images.” A Staff Engineer thinks of a CDN as a Distributed Edge Network that manipulates BGP, TCP, and TLS to cheat the speed of light.
1. The Physics of Latency
Light travels at 300 km/ms in a vacuum, but only ~200 km/ms in fiber.
- New York -> London: ~35ms one-way (best case).
- TCP Handshake (3-way): 1.5 Round Trips = ~105ms.
- TLS Handshake (2x RTT): ~140ms.
Before you send a single byte of data, you’ve wasted 250ms. CDNs solve this by terminating the connection closer to the user.
2. Anycast Routing (Magic IPs)
How does a user in Tokyo and a user in London both ping 1.1.1.1 and get a response in <10ms?
BGP Anycast.
- Unicast: One IP = One Server.
- Anycast: One IP = Advertised by 300 data centers simultaneously.
- Mechanism: The Internet’s routers (BGP) automatically send packets to the topologically closest destination.
[!WARNING] Route Flapping: Since BGP is stateless, a network glitch can cause packets to “teleport” from the London PoP to the Amsterdam PoP mid-connection. Modern CDNs use “Connection ID” tracking to handle this seamlessly.
3. Split TCP & TLS Termination
The most powerful feature of a CDN isn’t caching—it’s Connection Termination.
Without CDN (Long-Haul)
Client (Tokyo) <—————-> Origin (Virginia)
- High RTT (200ms).
- Result: TCP “Slow Start” takes seconds to ramp up speed. Packet loss is catastrophic (recovery takes full RTT).
With CDN (Split TCP)
Client (Tokyo) <–> Edge (Tokyo) <==========> Origin (Virginia)
- Leg 1 (Client <-> Edge):
- RTT: 5ms.
- Result: Handshakes complete instantly. TCP window scales immediately.
- Leg 2 (Edge <-> Origin):
- Pre-warmed Connection: The CDN keeps a persistent, high-throughput connection open to your Origin.
- Optimized Routing: Traffic flows over the CDN’s private backbone (Argo/Acceleration), avoiding public internet congestion.
4. Interactive Latency Budget
See where the time goes. Toggle Edge Acceleration to see the savings.
5. Programmable Edge (Workers)
Modern CDNs allow you to run code at the PoP.
- Auth: Validate JWT tokens at the edge. Reject invalid requests before they touch your cloud (DDoS protection + Cost saving).
- A/B Testing: Assign users to buckets based on Region/Cookie at the edge.
- Custom Routing: “If User is Gold, route to Premium Origin.”
[!TIP] The “Serverless” Realization: Many apps shouldn’t be “Client -> Server -> DB.” They should be “Client -> Edge Function -> PlanetScale/DynamoDB.” This removes the “Origin” bottleneck entirely.