[Python-Dev] PEP 289 - Generator Expressions (original) (raw)

[Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward

Andrew Koenig ark-mlist at att.net
Mon May 3 15:19:39 EDT 2004

> That leads me to wonder what exactly the rationale for generator > expressions is. The PEP says that "time, clarity, and memory are > conserved by using an generator expression" but I can only see how > memory is conserved. That is, I don't find them any easier to read than > list comprehensions and I don't understand the performance implications > very well. It's not obvious to me that their faster.

I've been skimming due to being out of town and catching up, but I haven't seen a direct response to Jeremy's question about the rationale. Jeremy, do you still want an answer?

I can see two potentially important cases where generator expressions win big over list comprehensions:

1) Where the code that is consuming the sequence yielded by the

generator expression terminates before consuming the entire sequence;

2) Where the code that is consuming the sequence is an online

algorithm, and there is a potential delay between generating elements of the sequence.

It is easier to find an example of the second case than of the first:

foo(bar(line) for line in sys.stdin)

If foo expects a generator as its input:

def foo(x):
    for line in x:
        print line

then using a generator expression instead of a list comprehension will cause the program to print each line of output after reading the corresponding line of input, rather than consuming all the input and then printing all the output.

More information about the Python-Dev mailing list