-
How a Neural Network Actually Learns: From a Single Neuron to Backpropagation
A single artificial neuron can’t solve XOR. Here’s the full, provable chain of fixes — nonlinearity, activation functions, initialization, backpropagation, batching, and the right loss function — that turns that…
-
Why Cross-Entropy Beats MSE for Classification: The Gradient Math, Proven (Ep:03.06)
A confidently wrong sigmoid+MSE network gets a weaker correction signal exactly when it needs a stronger one. Module 3 of From Zero to Agents derives why softmax+cross-entropy avoids this pathology,…
-
Backpropagation From Scratch: Deriving the Four Equations, Verified Against PyTorch (Ep:03.04)
Backpropagation is four equations, applied layer by layer. Module 3 of From Zero to Agents derives all four from the chain rule, implements them in raw NumPy with no autograd,…
-
The Vanishing Gradient Problem Explained: Why ReLU Replaced Sigmoid (Ep:03.02)
Sigmoid’s derivative maxes out at 0.25 and shrinks fast — multiply that through 15 layers and gradients vanish to nearly zero before reaching early layers. Module 3 of From Zero…
-
How Multi-Layer Neural Networks Solve XOR (and Why Nonlinearity Is Non-Negotiable) (Ep:03.01)
Stacking two perceptrons solves the problem one perceptron provably can’t. Module 3 of From Zero to Agents hand-builds a working XOR network, proves why linear layers alone can never work…
-
Derivatives and Gradients — The Mechanism Behind “Learning” (Ep:02.02)
Every neural network “learns” by following a gradient downhill. Module 2 of From Zero to Agents builds derivatives, partial derivatives, and the chain rule from first principles, verifies them against…
