Calculus Fundamentals

Welcome to Calculus Fundamentals! This module covers the mathematical engine that powers modern Machine Learning. Without calculus, we wouldn’t have Gradient Descent, Backpropagation, or deep neural networks.

1. Chapters

  1. The Rate of Change: Derivatives Explained
    • Understand the derivative as a measure of sensitivity.
    • Visualize tangent lines and learn about AutoDiff (Dual Numbers).
  2. The Toolbox: Rules of Calculus
    • Master the Power, Product, and Quotient rules.
    • Deep dive into the Chain Rule, the heart of Backpropagation.
  3. Multivariable Calculus: The Gradient Vector
    • Move from single-variable slopes to multi-dimensional gradients.
    • Understand Partial Derivatives, Jacobians, and Hessians.
  4. Approximation: The Taylor Series
    • Learn how to approximate complex functions with simple polynomials.
    • See the connection between Taylor Series, Gradient Descent, and Newton’s Method.
  5. Case Study: Gradient Descent
    • Visualize the learning algorithm of AI.
    • Experiment with Learning Rates, Momentum, and Saddle Points.
  6. Module Review: Flashcards & Cheat Sheet
    • Review key concepts and formulas.
    • Test your knowledge with interactive flashcards.

2. Learning Goals

By the end of this module, you will be able to:

  • Explain what a derivative is in the context of sensitivity analysis.
  • Compute derivatives using standard rules and understand how AutoDiff works.
  • Visualize and compute gradients for multivariable functions.
  • Understand how optimization algorithms like Gradient Descent navigate loss landscapes.