Protection Mechanisms

Security in an operating system is not just about passwords; it is about isolation. How do you prevent a buggy video game from crashing the entire system? How do you stop a web browser from reading your banking app’s memory?

The answer lies in the Hardware/Software Contract. The CPU itself enforces boundaries through Protection Rings.

1. The Ring Model (x86 Architecture)

The CPU operates in different “modes” or “rings” of privilege.

  • Ring 0 (Kernel Mode):
  • God Mode. Full access to all hardware instructions (IO ports, CPU control registers CR0, CR3).
  • The OS Kernel (Linux, Windows NT, macOS XNU) runs here.
  • A crash here causes a Kernel Panic or Blue Screen of Death.
  • Ring 1 & 2:
  • Historically designed for device drivers, but largely unused in modern OSs (drivers now often run in Ring 0 or Ring 3).
  • Ring 3 (User Mode):
  • Restricted Mode. Cannot access hardware directly.
  • Applications (Chrome, Discord, VS Code) run here.
  • Attempting to run a privileged instruction (like disabling interrupts CLI) causes a General Protection Fault (GPF).

The Mechanism: CPL vs DPL

Every memory segment has a Descriptor Privilege Level (DPL) (0-3). The CPU tracks the Current Privilege Level (CPL) (0-3) in the CS (Code Segment) register.

[!IMPORTANT] The Rule: Access is allowed only if CPL ≤ DPL. Ring 0 (CPL=0) can access Ring 3 data. Ring 3 (CPL=3) CANNOT access Ring 0 data.

2. Interactive: Protection Ring Simulator

Visualize how the CPU enforces privilege levels. Try to execute privileged instructions from User Mode.

CPU Mode
RING 3 (User)
Instruction Queue
> System initialized. > Running in Ring 3 (User Mode).

3. Sandboxing & Principle of Least Privilege

Even if code runs in Ring 3, it can still delete your files (because you own the files). To prevent this, we use Sandboxing and Privilege Dropping.

Example: Privilege Dropping

A web server needs to bind to Port 80 (privileged < 1024). It starts as root, binds the port, and then IMMEDIATELY drops to a nobody user.

Go
Java
package main

import (
	"fmt"
	"log"
	"syscall"
)

// Run this as root to see it work
func main() {
	// 1. Initially running as root (UID 0)
	uid := syscall.Getuid()
	fmt.Printf("Initial UID: %d (Root)\n", uid)

	// ... Bind Port 80 here ...

	// 2. Drop privileges to a normal user (e.g., UID 1000)
	// This is a one-way trip. You cannot go back to root easily.
	const nobodyUID = 1000
	const nobodyGID = 1000

	if err := syscall.Setgid(nobodyGID); err != nil {
		log.Fatalf("Failed to set GID: %v", err)
	}
	if err := syscall.Setuid(nobodyUID); err != nil {
		log.Fatalf("Failed to set UID: %v", err)
	}

	fmt.Printf("New UID: %d (Restricted)\n", syscall.Getuid())

	// 3. Try to do something privileged (will fail)
    // Note: In Go, we usually check errors rather than crashing
    // like a C segmentation fault.
}

import java.io.IOException;

public class SandboxExample {
    public static void main(String[] args) {
        // Java doesn't have direct syscall access to setuid like C/Go.
        // Instead, we use ProcessBuilder to spawn sandboxed processes.

        try {
            // Running a command as a specific (restricted) user
            // Requires 'sudo' configuration or running as root initially
            ProcessBuilder pb = new ProcessBuilder(
                "sudo", "-u", "nobody", "python3", "script.py"
            );

            pb.inheritIO();
            Process p = pb.start();

            int exitCode = p.waitFor();
            System.out.println("Sandboxed process exited with: " + exitCode);

        } catch (IOException | InterruptedException e) {
            e.printStackTrace();
        }
    }
}

4. Summary

  • Hardware enforces Security: Ring 0 vs Ring 3 is physical, not just software.
  • Context Switching: System calls allow Ring 3 code to ask Ring 0 to do work via a controlled “Gate”.
  • Defense in Depth: Use OS-level permissions (UIDs) on top of Hardware rings.