Implementing Automatic Differentiation in Pure Python

A recording of me explaining and implementing automatic differentiation in pure Python. I start with some mathematics of forward and reverse mode autodiff and then implement interleaved forward mode autodiff. Then I explain how to use Dual Numbers and implement forward mode autodiff using Dual Numbers. Finally, I construct a computation graph and perform a topological sort to achieve reverse mode autodiff.

Accompanying blog post with resources and links to code: https://kyscg.github.io/2025/05/18/autodiffpython

Видео Implementing Automatic Differentiation in Pure Python канала Crossing The Hellespont