[Python-Dev] Tricky way of of creating a generator via a comprehension expression (original) (raw)

Yury Selivanov yselivanov.ml at gmail.com
Wed Nov 22 10:56:02 EST 2017

On Wed, Nov 22, 2017 at 10:10 AM, Ivan Levkivskyi <levkivskyi at gmail.com> wrote:

On 22 November 2017 at 15:47, Paul Moore <p.f.moore at gmail.com> wrote:

I generally don't understand "await" in any context, so I deferred judgement on that :-) Based on your comment that they are equally tricky, I'd suggest we prohibit them both ;-) Less facetiously, comprehensions are defined in the language reference in terms of a source translation to nested loops. That description isn't 100% precise, but nevertheless, if yield/async in a comprehension doesn't behave like that, I'd consider it a bug. So current behaviour (for both yield and await) is a bug, and your proposed semantics for yield is correct. I think there may be a small misunderstanding here. The situation is different for comprehensions and generator expressions, let me summarize the current state: - yield in comprehensions works "wrong" (a shorthand for not according to the docs/naive expectations, i.e. not equivalent to for loop) - await in comprehensions works "right" - yield in generator expressions works "wrong" - await in generator expressions works "wrong"

"await" in generator expressions works correct. I explained it here: https://bugs.python.org/issue32113

To clarify a bit more:

r = (V for I in X)

is equivalent to:

def _(): for I in X: yield V r = _()

For synchronous generator expression:

r = (f(i) for i in range(3))

is really:

def _(): for i in range(3): yield f(i) r = _()

For an asynchronous generator expression:

r = (await f(i) for i in range(3))

is equivalent to:

def _(): for i in range(3): yield (await f(i)) r = _()

I'll update PEP 530 to clarify these transformations better.

After some thinking, both yield and await look quite mind bending in generator expressions, so maybe the right compromise strategy is: - fix yield in comprehensions - await in comprehensions already works - make both yield and await a SyntaxError in generator expressions.

I'm all for prohibiting using 'yield' expression in generator expressions or comprehensions. The semantics is way to hard to understand and hence be of any value.

Making 'await' a SyntaxError is absolutely not an option. Async generator expressions are a shorthand syntax for defining asynchronous generators (PEP 525), and it's already being used in the wild.

Yury

More information about the Python-Dev mailing list