The Symphony of Containers

[!NOTE] This module explores the core principles of The Symphony of Containers, deriving solutions from first principles and hardware constraints to build world-class, production-ready expertise.

1. The Problem: “Run Command Hell”

Imagine you are onboarding a new developer. You send them this slack message:

“Hey, to run the app, first pull mongo:5. Then run it on port 27017. Then build the backend image. Run that, but make sure you link it to the mongo container. Oh, and don’t forget to set MONGO_URI env var. Then build the frontend…”

This is imperative orchestration. You are describing how to do it, step-by-step. Think of this like trying to conduct a symphony orchestra by whispering individual instructions into the ear of every single musician, one at a time, while the concert is already happening.

The consequences of this approach are severe:

  1. Network Volatility: Manually bridging containers to talk to each other requires tracking internal, dynamic IP addresses that change on every reboot.
  2. Data Destruction: Forgetting a -v (volume) flag when running a database container means the moment the container stops, all user data vanishes.
  3. Tribal Knowledge: The exact order of operations exists only in the mind of the senior developer, creating a massive bus factor.

This manual method is error-prone, hard to share, and impossible to version control.

2. The Solution: Declarative Orchestration

Docker Compose allows you to define the desired state of your application in a single file: docker-compose.yaml.

Instead of running commands, you write a “recipe” for your infrastructure.

[!TIP] Compose vs Dockerfile

  • Dockerfile: Describes how to build a single image (the recipe for a cake).
  • Compose: Describes how to run multiple containers together (the menu for the dinner party).

3. Anatomy of docker-compose.yaml

A Compose file has three main sections:

  1. Services: The computing components (your app, database, cache).
  2. Networks: The communication channels.
  3. Volumes: The persistent storage.

Interactive: The Compose Architect

Click the buttons to add services to your stack and watch the YAML generate automatically.

Add Service

(Your stack is empty)
services:

4. The Implicit Default Network & Internal DNS

Before we look at versioning, you must understand Compose’s most powerful automatic feature: Internal DNS resolution via the default network.

When you run docker compose up, Docker creates a custom bridge network for the entire stack. Every service defined in the YAML file automatically joins this network.

The Magic Trick: Docker automatically injects DNS records into every container. The hostname of those records is precisely the name of the service defined in your YAML file.

If your backend needs to connect to the database, it doesn’t need to know the database’s dynamic IP. It simply connects to postgres://database:5432. Compose handles the DNS resolution internally, entirely isolated from the host machine’s physical network interface.

Architecture Traceability: The Compose Default Network

  • Host Machine eth0: Only exposed ports (e.g., 8080:80) bind to the host interface.
  • Docker Daemon bridge docker0: Creates the isolated app_default subnet.
  • Service backend (IP: 172.x.x.2): Sends request to hostname database.
  • Embedded DNS (127.0.0.11): Resolves database to 172.x.x.3.
  • Service database (IP: 172.x.x.3): Receives the request on internal port 5432.

4.5. The Versioning Confusion

You might see version: '3.8' or version: '2' in older files.

[!NOTE] The Modern Standard Since the Docker Compose Specification (2020+), the top-level version field is optional and essentially deprecated. Modern Compose (docker compose, note the lack of hyphen) simply ignores it and treats the file according to the latest spec.

We recommend omitting the version field for new projects.

5. A Real-World Example

Here is a typical stack: A Go backend, a React frontend, and a Postgres database.

services:
  # 1. Frontend Service
  frontend:
    build:
      context: ./client
      dockerfile: Dockerfile
    ports:
      - "80:80"      # Host:Container
    depends_on:
      - backend      # Start backend first
    restart: always  # Restart policy

  # 2. Backend Service
  backend:
    build: ./server  # Shorthand for context
    environment:
      - DB_HOST=database
      - DB_USER=postgres
    ports:
      - "8080:8080"
    depends_on:
      database:
        condition: service_healthy
    deploy:
      resources:     # Hardware Limits
        limits:
          cpus: '0.50'
          memory: 512M

  # 3. Database Service
  database:
    image: postgres:15-alpine
    environment:
      - POSTGRES_PASSWORD=secret
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  db_data:           # Named volume for persistence

Key Fields Explained

  1. build vs image:
    • build: Creates an image from a local Dockerfile.
    • image: Pulls a pre-built image from a registry (Docker Hub).
  2. ports: Maps Host_Port:Container_Port. Only needed if traffic is entering from outside the Compose network.
  3. volumes: Mounts storage. db_data is a named volume managed by Docker (survives container restart).
  4. depends_on: Controls startup order.
    • Basic: “Start DB before Backend”.
    • Advanced: “Wait until DB is actually healthy (responding to pings) before starting Backend”.
  5. restart: Hardware constraints and failovers. always ensures the container reboots if the host crashes.
  6. deploy.resources: Prevents the “Noisy Neighbor” problem by hard-capping CPU and memory usage via Linux cgroups.

6. Environment Management: .env vs environment

Do not commit raw passwords to your docker-compose.yaml. Compose natively supports reading from a .env file located in the same directory.

The environment block within the YAML can then interpolate those values:

  database:
    image: postgres:15
    environment:
      - POSTGRES_PASSWORD=${DB_PASSWORD} # Pulls from local .env

7. Lifecycle Commands & Hardware Realities

Understanding exactly what happens on the host machine is critical:

Command Hardware Reality Data Persistence
docker compose up -d Compiles YAML, creates isolated Linux bridge network, provisions named volumes on host disk, creates containers via cgroups. Starts fresh or resumes existing.
docker compose stop Sends SIGTERM to main process in containers. RAM is cleared, CPU limits dropped. Volumes and network remain untouched.
docker compose down Stops containers, deletes the isolated bridge network, and deletes the containers. Named volumes survive.
docker compose down -v The Nuclear Option. Drops containers, networks, AND physically deletes the named volumes from the host disk. Data is destroyed.

[!WARNING] Compose vs Kubernetes Docker Compose is ideal for local development, CI/CD pipelines, and single-server deployments. However, because it is bound to a single physical host machine (single Docker Daemon), it cannot handle high-availability scaling across multiple nodes. For distributed production orchestration, you need Kubernetes.

8. Why This Matters

By committing this file to Git, you guarantee that every developer on your team runs the exact same environment.

  • No “I forgot to install Postgres”.
  • No “Which version of Redis are we using?”.
  • No “It works on my machine”.

If it works in Compose, it works everywhere.