Build a Transformer (GPT-style)
Build a tiny GPT in NumPy: token + position embeddings, layer normalization, scaled dot-product self-attention, multi-head attention with causal mask, feed-forward networks, residual connections, sampling (greedy, top-k, top-p), and the architecture behind ChatGPT, Claude, Llama.
advancedFree4 hours14 lessons
Start learningor sign up to track progress
Curriculum
Loading reference solution…