Playing by Rules: Lagrange Multipliers
1. Introduction: The Fence
Usually, we minimize f(x). But what if we must stay on a path g(x) = c?
- Example: Minimize cost (f), but satisfy nutritional requirements (g).
- Example: Maximize Entropy (f), but probabilities must sum to 1 (g).
2. The Intuition
Imagine you are hiking up a hill f(x,y). You want to reach the highest point, but you must stay on a specific trail g(x,y)=0.
You walk along the trail.
- If the trail cuts across the contour lines of the hill, you can still go higher by moving along the trail.
- You stop when the trail runs parallel to the contour lines.
At this point, the gradient of the hill ∇f and the gradient of the trail ∇g are parallel!
3. The Lagrange Multiplier (λ)
The condition is:
∇f = λ∇g
We define a new function called the Lagrangian:
L(x, y, λ) = f(x, y) - λ(g(x, y) - c)
Finding where ∇L = 0 gives us the optimal point satisfying the constraint.
4. Interactive Visualizer: Constrained Climber
- Background: Hill height map (f(x,y)).
- Blue Line: Constraint (g(x,y) = x2 + y2 = 4, a circle).
- Yellow Dot: Move it along the constraint circle.
- Arrows:
- Red: Gradient of Hill (∇f).
- Green: Gradient of Constraint (∇g).
Find the point where the Red and Green arrows align (are parallel). That is the maximum!
Click/Drag on the Blue Circle. Align the Red and Green arrows.
5. Summary
- Constraint: Restricts the search space.
- Tangency: The optimal point is where the “objective slope” matches the “constraint slope”.
- Lagrange Multipliers: The math tool to find these points analytically.