[Python-Dev] namespace for generator expressions (original) (raw)

Phillip J. Eby pje at telecommunity.com
Mon Jan 26 17:36:19 EST 2004

At 09:02 AM 1/27/04 +1100, Delaney, Timothy C (Timothy) wrote:

> From: Guido van Rossum > > > I'm happy to see progress on the generator expressions > implementation, > > but I think the specification of namespaces, which is just a sketch, > > might be simplified. > > Ouch! Where were you when this PEP was discussed on python-dev? I > was originally strongly in your camp, but Tim and several others > convinced me that in every single case where a generator expression > has a free variable, you want early binding, not late.

I think I agree with Jeremy. I was originally in the early-binding camp, but I think we're better off trusting the programmer.

The intent is that listcomps should be "safely" replaceable with genexprs anywhere that an iterator is acceptable in place of a list.

How often is a generator expression not going to be evaluated almost immediately? I guess, when they're passed to a function. But even in that case, how often are the bindings going to change? Except in pathological cases, they won't.

A trivial example is:

iterators = [] for i in range(5): iterators.append(x*2 for x in range(i))

print map(list,iterators)

If a listcomp is used, you get:

[[],[0,],[0,2],[0,2,4],[0,2,4,6],[0,2,4,6,8]]

If genexprs do late binding, you get:

[[0,2,4,6,8],[0,2,4,6,8],[0,2,4,6,8],[0,2,4,6,8],[0,2,4,6,8]]

which is a significant difference in semantics. To make this code work with a late binding genexpr, it becomes necessary to create a function definition, thus negating the benefit of using the genexpr in the first place.

As to how common this is, ask how often you use a list comprehension inside of another loop, and then ask how often you'd use a genexpr instead if it was available. Finally, ask yourself whether you're likely to remember that you need to totally rewrite the structure if you decide to use a genexpr instead. :)

(Of course, I also often forget that free variables don't bind early in nested functions! It always seems strange to me that I have to write a function that returns a function in order to just define a function with bound variables. Indeed, it's hard to think of a time where I've ever wanted late binding of variables in a nested function.)

More information about the Python-Dev mailing list