Foundations of Neural Networks and Transformers · Lesson 5
How Neural Networks Work
The forward pass moves input through the layers to produce a prediction. The loss compares it to the target.
Backpropagation applies the chain rule from the output back to every weight, computing how much each weight contributed to the error. The optimiser then updates the weights.
Over many epochs the network learns internal features — early layers capture simple patterns and deeper layers combine them into abstract concepts.
Key points
- Forward pass produces predictions
- Backpropagation computes all gradients via the chain rule
- Optimiser updates weights
- Deeper layers learn more abstract features