[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part) (original) (raw)

Paul Moore p.f.moore at gmail.com
Wed Jun 27 03:30:00 EDT 2018

On 27 June 2018 at 07:54, Steven D'Aprano <steve at pearwood.info> wrote:

Comprehensions already run partly in the surrounding scope.

I tried to take a survey of people on the Python-List mailing list, so see what their expectations of comprehension scope was. Disappointingly, not many people responded, but those who did, invariably think in terms of comprehensions running inside their enclosing scope, like any other expression: https://mail.python.org/pipermail/python-list/2018-June/734838.html (Please excuse the doubled-up posts, some misconfigured news server is periodically sending duplicate posts.) (Oh and ignore my comment about Python 2 -- I was thinking of something else.) Given the code shown: def test(): a = 1 b = 2 result = [value for key, value in locals().items()] return result

nobody suggested that the result ought to be the empty list, which is what you should get if the comprehension ran in its own scope. Instead, they all expected some variation of [1, 2], which is what you would get if the comprehension ran in the enclosing scope. A decade or more since generator expressions started running in their own half-local-half-sublocal scope, people still think of scoping in terms of LEGB and don't think of comprehensions as running in their own scope except to the very limited degree that sometimes they are either surprised or pleased that "the loop variable doesn't leak".

But test() returns [1, 2]. So does that say (as you claim above) that "the comprehension ran in the enclosing scope"? Doesn't it just say that the outermost iterable runs in the enclosing scope?

So everybody expected the actual behaviour? (Disclaimer: in my response, I said that I had no clear expectation, which I stand by - locals() exposes implementation details that I don't normally feel that I need to know - but certainly the majority of respondents expected 1 and 2 to appear).

On the other hand,

def test2(): ... a = 1 ... b = 2 ... result = [locals().items() for v in 'a'] ... return result ... test2() [dict_items([('v', 'a'), ('.0', <str_iterator object at 0x0000015AA0BDE8D0>)])]

and I bet no-one would have expected that if you'd posed that question (I certainly wouldn't). Although some might have said [('v', 'a')]. I suspect some would have expected a and b to appear there too, but that's just a guess...

So yes, it's likely that people would have found the current behaviour unexpected in respect of locals(). But I imagine most people only care about the effective results when referencing variables, and

def test3(): ... a = 1 ... b = 2 ... result = [a for v in (1,)] ... return result ... test3() [1]

i.e., thanks to scope nesting, you can still reference locals from the enclosing scope.

The problem is that := allows you to change values in a scope, and at that point you need to know which scope. So to that extent, the locals() question is important. However, I still suspect that most people would answer that they would like := to assign values as if they were in the enclosing scope, which is not really something that I think people would express in answer to a question about locals(). This can be achieved with an implicit "nonlocal" (and some extra shenanigans if the enclosing scope has a nonlocal or global declaration itself). Which, AIUI, is what the current proposal tries to do.

IMO, the big question over the current PEP 572 proposal is whether it goes too far in the direction of "do what I mean". Superficially, the semantics are pretty clearly "what people would expect", and indeed that's been the whole focus recently to capture and satisfy expected behaviour. But there are edge cases (there always are when you work from messy real-world requirements rather than nice clean mathematical definitions ;-)) and the question is essentially whether any of those are bad enough to be an issue.

I'm starting to feel that they aren't, and I'm moving towards a cautious +0 (or even +1) on the proposal. Paul

More information about the Python-Dev mailing list