[Python-Dev] generator expression syntax (original) (raw)

Gareth McCaughan gmccaughan at synaptics-uk.com
Wed Mar 24 04:37:45 EST 2004

On Wednesday 2004-03-24 05:46, Robert Mollitor wrote:

The idea of generator expressions seems good, but the proposed syntax seems a little wrong to me.

First, the syntax is too dependent on the parentheses. To my mind, this is a fourth meaning for parentheses. (1) Parentheses group and associate expressions: "(a * (b + c))", "if (a and b)". (2) Parentheses construct tuples: "(1, 2, 3)", "()", "('a',)". (3) Parentheses enclose argument lists (arguably a special case of tuple-constructor): "def f(a, b, c)", "obj.dump()", "class C (A, B)". And now (4*) generator expressions: "(x*x for x in list)". I realize that in some sense the parentheses are not part of the expression syntax (since we wouldn't require "sum((x * x for x in list))"), but they are mandatory nonetheless because you can't have "a = x*x for x in list". This seems like it stretching a familiar construct too far.

Actually 4 and 2 are very similar. It's commas that construct tuples, not parentheses; the parentheses are just necessary in some contexts to shield the tuple construction from other things going on nearby. Requiring parens around a generator expression is just like requiring parens around a single-element tuple expression.

Second, it looks like a "tuple comprehension". The list comprehension construct yields a list. A generator expression looks like it should yield a tuple, but it doesn't.

OK, so generator expressions should be wrapped in the same punctuation that's used to write explicit generators. Er, except that there isn't any.

Third, it seems Lisp-ish or Objective-C-ish and not Pythonic to me. I realize that is just a style thing, but that's the flavor I get.

There are two separate statements here: (1) it seems Lisp-ish or ObjC-ish, and (2) it doesn't seem Pythonic. #1 isn't a problem in itself; are you deducing #2 from #1 or claiming that it's true in its own right? (I don't see either #1 or #2 myself, but I find #1 easier to sympathize with than #2.)

Fourth, it seems like this variable binding thing will trip people up because it is not obvious that a function is being defined. Lambdas have variable binding issues, but that is obviously a special construct.

It's not clear to me that lambdas are any more obviously a special construct than generator expressions. Are you sure it's not just that you're used to lambdas and know that they're special, but aren't yet used to generator expressions?

OK, I not completely sure if this will work to everyone's satisfaction, but here is my proposal: replace the

(x*x for x in a) construct with lambda: yield x*x for x in a

I predict that to a good first approximation this one won't work to anyone's satisfaction.

CONS - "Ick, lambdas" - It's longer

PROS

- Lambdas are funky, but they are explicitly funky: look up 'lambda' in the index and go to that section of the book

This "pro" would equally justify overloading "if" or inventing a keyword "wombiquangle" for this purpose.

- Use the variable binding rules of lambas and people will be as happy with that as they are with lambdas in the first place (for better or worse)

The use cases of lambdas and of generator expressions are not the same. Why should the appropriate variable binding rules be the same? And why, therefore, should people be equally happy with them?

So here would be the recasting of some of examples in PEP 289:

sum(lambda: yield x*x for x in range(10)) d = dict (lambda: yield x, func(k) for k in keylist) [etc]

I can't believe you typed all those in and looked at them and still like this idea :-).

# I think the following would work, too for xx in lambda: yield x*x for x in range(10): print xx

# If so, how's this for decadent for i,a in lambda: yield i,list[i] for i in range(len(list)): print i, a

You think this is a good thing? Compare these, with their misleading initial segments "for something in lambda:" and all, with

for xx in (x*x for x in range(10)):
    print x

for i,a in ((i,list[i]) for i in range(len(list))):
    print i,a

which, for all their allegedly superfluous parentheses and allegedly weird variable binding behaviour, I can at least read.

-- g

More information about the Python-Dev mailing list