[Python-Dev] Tricky way of of creating a generator via a comprehension expression (original) (raw)

Ivan Levkivskyi levkivskyi at gmail.com
Wed Nov 22 14:01:52 EST 2017

On 22 November 2017 at 18:15, Paul Moore <p.f.moore at gmail.com> wrote:

On 22 November 2017 at 16:47, Ivan Levkivskyi <levkivskyi at gmail.com> wrote: > On 22 November 2017 at 17:43, Paul Moore <p.f.moore at gmail.com> wrote: >> >> On 22 November 2017 at 16:30, Ivan Levkivskyi <levkivskyi at gmail.com> >> wrote: >> > On 22 November 2017 at 17:24, Antoine Pitrou <solipsis at pitrou.net> >> > wrote: >> >> Given a comprehension (e.g. list comprehension) is expected to work >> >> nominally as constructor(generator expression) >> > >> > As Yury just explained, these two are not equivalent if there is an >> > await >> > in the comprehension/generator expression. >> >> As Antoine said, people expect them to work the same. > > > The difference is that a generator expression can be used independently, one > can assign it to a variable etc. not necessary to wrap it into a list() > Anyway, can you propose an equivalent "defining" code for both? Otherwise it > is not clear what you are defending.

[...] 1. List comprehensions expand into nested for/if statements in the "obvious" way - with an empty list created to start and append used to add items to it. 1a. Variables introduced in the comprehension don't "leak" (see below). 2. Generator expressions expand into generators with the same "nested loop" behaviour, and a yield of the generated value. 3. List comprehensions are the same as list(the equivalent generator expression). Paul, OK, I think how to formulate these rules more "precisely" so that they will be all consistent even if there is a yield inside. The key idea is that neither comprehensions nor generator expressions should create a function scope surrounding the expr in (expr for ind in iterable) and [expr for ind in iterable]. (this still can be some hidden implementation detail)

So as Serhiy proposed in one of his first posts any comprehensions and generator expressions with yield are not valid outside functions. If such comprehension or generator expression is inside a function, then it makes it a generator, and all the yileds are yielded from that generator, for example:

def fun_gen(): return list((yield i) for i in range(3))

should work as following:

g = func_gen()

g.send(42) 0 g.send(43) 1 g.send(44) 2 try: g.send(45) except StopIteration as e: assert e.value == [42, 43, 44]

And exactly the same with

def fun_comp(): [(yield i) for i in range(3)]

I hope with this we can close the no-async part of the problem. Currently this is not how it works, and should be fixed. Do you agree?

The async part can then be considered separately.

-- Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20171122/d567145a/attachment-0001.html>

More information about the Python-Dev mailing list