Skip to content

Build a Neural Network

Build an MNIST classifier from scratch in NumPy: neurons, dense layers, sigmoid/ReLU/softmax, MSE/cross-entropy loss, manual backpropagation, gradient descent + Adam, training loop, and the path to PyTorch/TensorFlow.

advancedFree6 hours25 lessons
Start learningor sign up to track progress

Curriculum

Loading reference solution…

Going further

Read

  • Deep Learning (Goodfellow, Bengio, Courville) — the canonical textbook. Free online at deeplearningbook.org.
  • 3blue1brown's neural networks playlist — the best visual explanation of backprop.
  • Andrej Karpathy's Zero to Hero lecture series — code-along that picks up exactly where this course ends.
  • Pattern Recognition and Machine Learning (Bishop) — older but rigorous on the math.

Build next

  • Convolutional layers: replace the dense MLP with Conv2D + MaxPool for image data. Push MNIST accuracy past 99%.
  • A real autograd engine: instead of hand-coding backward() per layer, build a tape-based autodiff. ~150 LOC. See micrograd by Karpathy.
  • Batch normalization, dropout, weight decay: the regularization toolkit.
  • GPU: port to CUDA via CuPy or PyTorch.

Continue with our courses

  • Build a Transformer — same backprop machinery, much more interesting architecture.
Build a Neural Network