Skip to content

Build a Transformer (GPT-style)

Build a tiny GPT in NumPy: token + position embeddings, layer normalization, scaled dot-product self-attention, multi-head attention with causal mask, feed-forward networks, residual connections, sampling (greedy, top-k, top-p), and the architecture behind ChatGPT, Claude, Llama.

advancedFree4 hours14 lessons
Start learningor sign up to track progress

Curriculum

Loading reference solution…
Build a Transformer (GPT-style)