Skip to content

Build a Neural Network

Build an MNIST classifier from scratch in NumPy: neurons, dense layers, sigmoid/ReLU/softmax, MSE/cross-entropy loss, manual backpropagation, gradient descent + Adam, training loop, and the path to PyTorch/TensorFlow.

advanced25 lessons8 chapters25 graded exercisesPython
Start course — it’s freeWork in your own editor

No sign-up needed for lesson 1 · certificate on completion · sign up to save progress

What you’ll have built

Chapter by chapter. Every step is a graded exercise.

  • Foundations
  • Forward & Backward
  • Training
  • Production
  • Optimizers & Autograd
  • Regularization & Normalization
  • Architectures
  • Attention & Transformers
Starts in Python — solve in the language you choose in the editor, same tests either way.

Curriculum

8 chapters, 25 lessons. Each lesson is a short read, one graded exercise, and a quiz.

  1. 1What Neural Networks SolveRead · exercise · quiz
  2. 2A Single NeuronRead · exercise · quiz
  3. 3Layers as Matrix OperationsRead · exercise · quiz
Loading reference solution…
Going further

When the tests are green, keep going.

Read

  • Deep Learning (Goodfellow, Bengio, Courville) — the canonical textbook. Free online at deeplearningbook.org.
  • 3blue1brown's neural networks playlist — the best visual explanation of backprop.
  • Andrej Karpathy's Zero to Hero lecture series — code-along that picks up exactly where this course ends.
  • Pattern Recognition and Machine Learning (Bishop) — older but rigorous on the math.

Build next

  • Convolutional layers: replace the dense MLP with Conv2D + MaxPool for image data. Push MNIST accuracy past 99%.
  • A real autograd engine: instead of hand-coding backward() per layer, build a tape-based autodiff. ~150 LOC. See micrograd by Karpathy.
  • Batch normalization, dropout, weight decay: the regularization toolkit.
  • GPU: port to CUDA via CuPy or PyTorch.

Continue with our courses

  • Build a Transformer — same backprop machinery, much more interesting architecture.