[Python-Dev] PEP 289 - Generator Expressions (original) (raw)

[Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward

Paul Prescod paul at prescod.net
Sat Apr 24 05:51:36 EDT 2004

Jeremy Hylton wrote:

...

That leads me to wonder what exactly the rationale for generator expressions is. The PEP says that "time, clarity, and memory are conserved by using an generator expression" but I can only see how memory is conserved. That is, I don't find them any easier to read than list comprehensions and I don't understand the performance implications very well. It's not obvious to me that their faster.

I think that there is a robustness argument to be made as well. It is very common to run into programs that work really well with small data sets and then completely run out of steam with large ones. Sometimes the algorithm is at fault. But it could also be that some Python programs fail to scale primarily because of the wastefulness of listcomps. Whether or not this is true in practice, it is certainly true that when you write programs that are meant to scale, you must consider every listcomp and think whether the dataset is going to get large or not. If so, you must switch to some more obfuscated syntax.

Paul Prescod

More information about the Python-Dev mailing list