Overlay Network

Bridge and Host networks work great on a single machine. But in a production cluster (like Docker Swarm or Kubernetes), containers run on different physical servers.

How does a container on Server A talk to a container on Server B as if they were on the same LAN? The answer is the Overlay Network.

1. The Magic of VXLAN

Docker uses VXLAN (Virtual Extensible LAN) to create a virtual network on top of the physical network.

First Principles: Encapsulation

  1. Original Packet: Container A sends a packet to Container B (10.0.0.5).
  2. Encapsulation: The Docker Engine on Server A wraps this packet inside a UDP packet.
  3. Transport: The UDP packet travels over the physical network (the “Underlay”) to Server B’s physical IP.
  4. Decapsulation: Server B receives the UDP packet, unwraps it, and delivers the original packet to Container B.

2. Interactive: VXLAN Encapsulation

Visualize how a packet traverses the physical network.

Node A (Physical: 192.168.1.10)
Container A
10.0.0.3
VTEP (Encapsulator)
UDP 192.168.1.20
IP 10.0.0.5
Node B (Physical: 192.168.1.20)
Container B
10.0.0.5
VTEP (Decapsulator)
Ready to transmit...

3. Creating an Overlay Network

To use Overlay, you typically need Docker Swarm initialized.

# Initialize Swarm on Node A
docker swarm init

# Create the overlay network
docker network create -d overlay my-multi-host-net

# Create a service on this network
docker service create --name web --network my-multi-host-net --replicas 2 nginx

Now, containers on different nodes can ping each other using the hostname web.

4. Summary

  • Overlay: Connects containers across multiple hosts.
  • VXLAN: The tunneling protocol used to encapsulate packets.
  • Swarm: The orchestrator that manages the overlay network state.