Backpropagation in plain Python
Implementing a neural network forward and backward pass from scratch with nothing but Python lists. No NumPy, no frameworks. Understanding the basics first.
PyTorch and TensorFlow handle backpropagation automatically, which makes it easy to train neural networks without understanding what's actually happening. This post implements the full forward and backward pass of a small two-layer network using only Python's built-in math library, no NumPy, no frameworks, to make the mechanics concrete.
It walks through the chain rule step by step: computing gradients for output weights, hidden layer weights, and biases; accumulating gradients across multiple hidden units; and performing the weight update. Frameworks abstract all of this, but when training goes wrong (vanishing gradients, exploding gradients, loss that doesn't move), understanding what's being computed is the difference between debugging and guessing. Published on Analytics Vidhya, January 2021.