[Python-Dev] Tricky way of of creating a generator via a comprehension expression (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Thu Nov 23 19:50:56 EST 2017

On 23 November 2017 at 23:04, Ivan Levkivskyi <levkivskyi at gmail.com> wrote:

I don't see why this particular case qualifies for such a radical measure as an exception to syntactic rules, instead of just fixing it (sorry Nick :-)

I've posted in more detail about this to the issue tracker, but the argument here is: because making it behave differently from the way it does now while still hiding the loop iteration variable potentially requires even more radical revisions to the lexical scoping rules :)

If somebody can come up with a clever trick to allow yield inside a comprehension to jump levels in a relatively intuitive way, that would actually be genuinely cool, but the lexical scoping rules mean it's trickier than it sounds.

Now that I frame the question that way, though, I'm also remembering that we didn't have "yield from" yet when I wrote the current comprehension implementation, and given that, it may be as simple as having an explicit yield expression in a comprehension imply delegation to a subgenerator.

If we went down that path, then a list comprehension like the following:

results = [(yield future) for future in list_of_futures]

might be compiled as being equivalent to:

def __listcomp_generator(iterable):
    result = []
    for future in iterable:
        result.append((yield future))
    return result

results = yield from _listcomp_generator(list_of_futures)

The only difference between the current comprehension code and this idea is "an explicit yield expression in a comprehension implies the use of 'yield from' when calling the nested function".

For generator expressions, the adjustment would need to be slightly different: for those, we'd either need to prohibit yield expressions, or else say that if there's an explicit yield expression present anywhere, then we drop the otherwise implied yield expression.

If we went down the latter path, then:

gen = ((yield future) for future in list_of_futures)

and:

gen = (future for future in list_of_futures)

would be two different ways of writing the same thing.

The pay-off for allowing it would be that you could write things like:

gen = (f(yield future) for future in list_of_futures)

as a shorthand equivalent to:

def gen(list_of_futures=list_of_futures):
    for future in list_of_futures:
        f(yield future)

(Right now, you instead get "yield f(yield future)" as the innermost statement, which probably isn't what you wanted)

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20171124/18e84150/attachment-0001.html>

More information about the Python-Dev mailing list