Issue 3331: Possible inconsistency in behavior of list comprehensions vs. generator expressions (original) (raw)

Compare the following behaviors:

Python 3.0a5 (r30a5:62856, May 10 2008, 10:34:28)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more 

information. >>> def f(x): ... if x > 5: raise StopIteration ... >>> [x for x in range(100) if not f(x)] Traceback (most recent call last): File "", line 1, in File "", line 1, in File "", line 2, in f StopIteration >>> list(x for x in range(100) if not f(x)) [0, 1, 2, 3, 4, 5]

One might object that the behavior of the list comprehension is identical to that of a for-loop:

>>> r = []
>>> for x in range(100):
...  if not f(x):
...   r.append(x)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<stdin>", line 2, in f
StopIteration

However, it can be argued that in Python 3 list comprehensions should be thought of as "syntatic sugar" for list(generator expression) not a for-loop with an accumulator. (This seems to be the motivation for no longer "leaking" variables from list comprehensions into their enclosing namespace.)

One interesting question that this raises (for me at least) is whether the for-loop should also behave like a generator expression. Of course, the behavior of the generator expression can already be simulated by writing:

>>> r = []
>>> for x in range(100):
...  try:
...   if f(x):
...    r.append(x)
...  except StopIteration:
...   break
... 
>>> r
[0, 1, 2, 3, 4, 5]

This raises the question, do we need both a break statement and raise StopIteration? Can the former just be made into syntatic sugar for the later?

You seem to be suggesting that a StopIteration raised in the body of a for-loop should cause the loop to break. That isn't (as far as I know) the point of StopIteration. StopIteration is only used to break the for-loop when raised by the iterator, not the body.

Hence I think the list comprehension is behaving correctly, as the for-loop is, in that they are both raising the StopIteration you threw, not catching it. That's valid, because you didn't throw it in the iterator, you threw it in the condition.

What's more strange (to me) is the fact that the generator expression stops when it sees a StopIteration. Note that this also happens if you do it in the head of the generator expression. eg

def f(x): if x > 5: raise StopIteration return x

list((f(x) for x in range(0, 100))) [0, 1, 2, 3, 4, 5]

However, if you translate that into the full generator function version:

def my_generator_expr(): for x in range(0, 100): yield f(x)

You see that it is behaving correctly.

So I think you discovered an interesting quirk, but it's hard to say anything here is misbehaving.

By the way this is not a new issue with Python 3.0. Flagging it as a Python 2.5 issue as well.