Container Runtime Hierarchy

[!NOTE] This module explores the core principles of Container Runtime Hierarchy, deriving solutions from first principles and hardware constraints to build world-class, production-ready expertise.

1. The Great Split

Before 2016, Docker was a monolith. One binary (dockerd) did everything: downloading, unpacking, networking, and running containers.

Today, it is a modular stack of specialized tools.

The Stack

1. Docker CLI: The user interface. Sends HTTP JSON to the daemon.
2. Dockerd: The high-level manager. Handles images, volumes, and networks.
3. Containerd: The industry standard container runtime. Manages the lifecycle (Start/Stop/Kill).
4. Containerd-Shim: A tiny process that sits between containerd and runc. It allows containerd (and Docker) to restart without killing running containers.
5. Runc: The low-level runtime. It spawns the process, sets up Namespaces/Cgroups, and exits.

---

2. Interactive: Runtime Flow Simulator

Trace the path of a docker run command.

Docker CLI

→

Dockerd

→

Containerd

→

Shim

→

Runc

→

Kernel

Waiting for command...

---

3. Code Examples

1. Go Implementation (Containerd Client)

This is how Kubernetes (via CRI) talks to containerd directly, bypassing Docker entirely.

package main

import (
	"context"
	"fmt"
	"github.com/containerd/containerd"
	"github.com/containerd/containerd/namespaces"
)

func main() {
	// 1. Connect to Containerd Socket
	client, err := containerd.New("/run/containerd/containerd.sock")
	if err != nil {
		panic(err)
	}
	defer client.Close()

	ctx := namespaces.WithNamespace(context.Background(), "default")

	// 2. Pull Image
	image, err := client.Pull(ctx, "docker.io/library/redis:alpine", containerd.WithPullUnpack)
	if err != nil {
		panic(err)
	}

	// 3. Create Container (Metadata)
	container, err := client.NewContainer(ctx, "redis-server", containerd.WithNewSnapshot("redis-snapshot", image))
	if err != nil {
		panic(err)
	}

	// 4. Create Task (Process) -> Calls Runc
	task, err := container.NewTask(ctx, containerd.Stdio)
	if err != nil {
		panic(err)
	}

	fmt.Printf("Task PID: %d\n", task.Pid())
	task.Start(ctx)
}

2. Java Implementation (Docker Socket)

Docker exposes a Unix Domain Socket at /var/run/docker.sock. Java can talk to it directly using HTTP. This is how tools like Testcontainers work.

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.UnixDomainSocketAddress; // Java 16+

public class DockerClient {

    public static void main(String[] args) throws Exception {
        // 1. Create HTTP Client over Unix Socket
        HttpClient client = HttpClient.newBuilder().build();

        // 2. Construct Request (List Containers)
        // Note: Java 16 introduced UnixDomainSocketAddress
        // For older Java, use a library like 'junixsocket'

        HttpRequest request = HttpRequest.newBuilder()
            .uri(new URI("http://localhost/containers/json"))
            .header("Content-Type", "application/json")
            .GET()
            .build();

        // This is pseudo-code for the transport layer
        // In reality, you need a custom BodyPublisher for Unix Sockets
        System.out.println("Sending GET /containers/json to Docker Daemon...");

        // Response would be JSON: [{"Id": "a1b2...", "Image": "redis", ...}]
    }
}

[!NOTE] Why the Shim? The shim allows containerd to exit (for upgrades) without killing the containers it started. The shim becomes the new parent of the container process, holding its STDIN/STDOUT open.

4. First Principles: Why Layers?

Why separate dockerd from containerd from runc?

1. Standardization (OCI): runc is the reference implementation of the OCI Runtime Spec. Anyone can write a replacement (e.g., kata-runtime for VM isolation, gvisor for sandboxing).
2. Stability: The massive Docker Daemon can crash or upgrade, but containerd and the shims keep the containers running.