Definition
Backpropagation applies the chain rule of calculus from the loss backward through each layer, producing gradients for every weight efficiently.
Without it, training deep networks would be computationally impractical.
In simple terms
If the final answer is wrong, backprop walks back through every step of the calculation asking "how much did you contribute to this mistake?"
Where you see it
- Every PyTorch or TensorFlow training step uses automatic differentiation (backprop under the hood).
- Fine-tuning LLMs updates weights via backpropagated gradients.
How it works
1.Forward pass
Compute predictions and loss.
2.Backward pass
Propagate gradients layer by layer.
3.Update
Optimizer steps weights using those gradients.
Why it matters
- Backprop is why deep learning scales — it made multilayer networks trainable.
Often confused
You must implement backprop by hand.
Frameworks compute gradients automatically; understanding the idea still helps debug training.