Generator Expressions

You will build lazy pipelines in one expression, feed them to sum, next or groupby, and avoid traps that give a wrong answer without raising anything. A list comprehension builds its whole list now; swap the brackets for parentheses and you get the kind of generator you wrote by hand with yield.

Parentheses, and the sole-argument rule

When the expression is the only argument, the call's parentheses double as its own. Add a second argument and you must wrap it yourself. sum takes an optional second argument, the start value: the total begins there instead of at 0.

What consumes one

A consumer is anything that pulls values from a stream: sum, min, max, sorted, set, str.join, next, a for loop. None needs a list first. any and all short-circuit: they stop pulling at the first decisive value, so the rest is never computed. So any(len(w) > 4 for w in words) stops at "banana", the third word, and never looks at the rest, while any([len(w) > 4 for w in words]) builds all six booleans before any sees one.

First match with next, then keep going

next(iterator, default) over a generator expression is a one-line search. enumerate(words, start=1) numbers items from 1 instead of 0:

A named iterator is an iterator stored in a variable. A generator expression over it copies nothing, it pulls from that same iterator, so each consumer leaves it where it stopped and the next carries on from there:

next pulls 4, 8 (rejected) and 15 (match, stop), leaving 16, 23, 42 for sum. The 4 and 8 are gone for good; a fresh iter(...) would start from 4 again.

Pipelines and itertools

Each stage pulls from the one before it, and nothing runs until list asks:

isdigit is true only for the characters 0 to 9, so "-5".isdigit() is False.

The itertools module composes streams without building lists:

groupby(stream, key=f) yields (key, group) pairs, where group is itself a lazy iterator over the items sharing that key. It only merges adjacent items, so sort by the same key first. With no key, the items themselves are compared.

A group is only valid until groupby moves on, so size it (list(g) or sum(1 for _ in g)) while the loop or comprehension is advancing groupby. Collect the pairs first and every group is empty: [(k, list(g)) for k, g in list(groupby("aab"))] gives [('a', []), ('b', [])].

The traps

One pass only. A spent generator is empty, and sum of nothing is 0, so a second sum(g) reports 0 silently.

Late lookup. Only the outermost iterable is evaluated immediately; every other name is read when the value is pulled. After rate = 2, g = (n * rate for n in range(3)) and then rate = 10, list(g) is [0, 10, 20], not [0, 2, 4].

No len, no indexing. len(g) raises TypeError. Use a list for a count, random access or a second pass.

Common mistakes

• A list comprehension where the stream must stay lazy. [int(t) for t in it] converts every remaining item (raising ValueError on "n/a") and drains it, so the next consumer sees nothing and reports 0.
• next(...) without a default on a stream with no match raises StopIteration.
• sorted before groupby when order matters: it merges equal items that were far apart into one group, so a count of runs comes out too low.
• list(groupby(...)) and then sizing the groups reports 0 for every one of them.

Your exercise

Audit a stream of lines held in one shared iterator: locate the first value above a threshold, then summarise what is left unread, including its runs of repeated lines. The usual failure is a list comprehension or a second iterator, which parses lines that must stay untouched.