Skip to content
Generators and yield
step 1/7

Reading — step 1 of 7

Learn

~3 min readGenerators and Iterators

Imagine you need to process a 10GB log file but your server has only 4GB of RAM. Or you need Fibonacci numbers but don't know how many ahead of time. Or you're piping data through several transformations and don't want to materialize an intermediate list of millions.

Generators solve all of these. A generator is a function that produces values one at a time, pausing between each one. It computes the next value only when you ask for it. Memory usage stays constant regardless of how many values it generates.

yield — turn a function into a generator

python

A function that uses yield becomes a generator function. Calling it doesn't run the body — it returns a generator object that runs the body lazily, yielding values one at a time.

How yield works mechanically

python

Key insight: each next() runs the body UNTIL the next yield, hands you that value, and pauses. The function's local state (variables, where it left off) is preserved between calls.

When the function finishes (or hits return), next() raises StopIteration. A for loop catches that automatically.

Why generators are amazing for memory

python

Real-world example: process a huge file line by line

python

The data flows through generators one item at a time — laptop-friendly even for files larger than RAM.

yield from — delegate to another generator

python

yield from gen yields every value from another iterable. Cleaner than nested for-loops.

Generators vs lists

GeneratorList
Lazy — values on demandEager — all values upfront
Constant memoryMemory grows with size
Iterate ONCEIterate any number of times
Can't index gen[5]Can index lst[5]
Can't len()Can len()

Common mistakes

  • Trying to use a generator twice: a generator is exhausted after one full iteration. Re-call the function for a fresh one.
  • return value from a generator: returning a value sets StopIteration.value, but most code never reads it. For early termination, just return.
  • Forgetting that the function body doesn't run on call — only on the first next() (or first iteration of a for-loop).

Discussion

Ask a question, share an insight, or help someone who’s stuck.

Sign in to post a comment or reply.

Loading…