[Python-Dev] Looking for examples: proof that a list comp is a function (original) (raw)

Tim Peters tim.peters at gmail.com
Tue May 15 18:30:22 EDT 2018

[ Tim, about the most version of the docs at https://docs.python.org/dev/reference/expressions.html#displays-for-lists-sets-and-dictionaries ]

I say "pretty much" because, for whatever reason(s), it seems to be trying hard not to use the word "function". But I can't guess what "then passed as an argument to the implicitly nested scope" could possibly mean otherwise (it doesn't make literal sense to "pass an argument" to "a scope").

[Nick Coghlan <ncoghlan at gmail.com>]

I think my motivation was to avoid promising exact equivalence with a regular nested function, since the define-and-call may allow us opportunities for optimization that don't exist when those two are separated (e.g. Guido's point in another thread that we actually avoid calling "iter" twice even though the nominal expansion implies that we should). However, you're right that just calling it a function may be clearer than relying on the ill-defined phrase "implicitly nested scope".

Plus that, as noted, what passing an argument "to a scope" means is mysterious.

Language standard committees struggle for years with how to phrase things so that no more than is intended appears to be promised. It's hard! For example, if you were to show a workalike function and note that the exact placement - and number - of iter() calls is not guaranteed, someone else would point out that you need to explicitly say that by "iter" you mean the builtin function of that name, not one user code may have overridden it with in the current scope. Then someone else will note that it's tedious to say things like that whenever they're needed, and more-general text will be added elsewhere in the docs saying that the rest of the docs always mean the language-supplied versions of such-&-such explicitly named functions/classes/modules/...

I'd say "nested function" anyway ;-) And for another reason: not just someone from Mars is prone to misreading 'scope", but just about anyone on Earth coming from another language. The idea that the word "scope" all by itself implies "and in general any name bound to within the top-level code spanned by the scope is implicitly local to the scope unless explicitly declared global or nonlocal in the scope" may be unique to Python.

For Chris's actual question, this is part of why I think adding "parentlocal" would actually make the scoping proposal easier to explain, as it means the name binding semantics aren't a uniquely magical property of binding expressions (however spelled), they're just a new form of target scope declaration that the compiler understands, and the binding expression form implies. Note: easier, not easy ;)

Adding an explanation of parentlocal to the docs could be a useful pedagogical device, but I don't think I'd support adding that statement to the language. It's too weird, and seems to be at a wrong level for plausible future language developments.

Let's step waaaaay back for a minute. In many languages with full-blown closures, first-class functions, and nested lexical scopes, it's pretty common to define the meaning of various language constructs in terms of calling derived lexically nested functions. In those languages, any "work variables" needed by the synthetic functions are declared as being local to those functions, and that's the end of it. They're done. All other names inside the expansions mean exactly the same as what they mean in whatever chunks of user-supplied code the construct interpolates into the synthesized functions. It doesn't matter one whit in which context(s) they appear.

That's the only long-term sane way to go about defining constructs in terms of calling synthesized functions interpolating user-supplied pieces of code.

Now if Python had been able to do that, the meaning of genexps and listcomps would have been defined, from the start, in terms of synthesized functions that declared all & only the for-target names "local". And, in fact, the change I'm suggesting wouldn't have required changing the comprehension implementation at all when assignment expressions were added. Instead the implementation would need to change to add assignment expression targets to the things declared local if it was decided that those targets should be local to the derived functions instead.

That's why this all seems so bloody obvious to me ;-) It's how virtually every other language in the business of defining constructs in terms of nested synthesized functions works.

So if that's something we may ever do again - and possibly even if we don't expect to ever do it again - I suggest a more generally useful approach would be to add a new flavor of function to Python. Namely one wherein the only locals are the formal arguments and those explicitly declared local. Whether or not a name is bound in the body would be irrelevant. To avoid a new keyword, local could be spelled not nonlocal ;-)

Note that the only use for parentlocal so far is tediously emulating the effects of what that hypothetical deflocal flavor of function would do all the time with names not declared local in it. If we ever do something like this again, it would be far easier and clearer to just say the synthetic functions are of the deflocal flavor, and here are the names declared local in this case: x, y, z, ...

User-defined functions may well find that useful at times too. Although it would be a large conceptual addition to part of Python, adding parentlocal would be too, and all by itself the latter looks like an incoherent pile of bizarre tricks. The meaning of deflocal would be immediately clear to people coming from any number of other modern-ish languages.

It also occurs to me that we could do something pretty neat for class scopes: have parent local declarations in methods target the implicit lexical scope where class lives (to support zero-arg super), not the class body. That would entail adding a "classlocal" declaration to target that implied scope, though.

That would give the following definition for "lexical scopes that parent local scoping can target": - module globals (parentlocal -> global) - function locals, including lambda expression locals (parentlocal -> nonlocal)

Except that for top-level functions, parentlocal -> global; and regardless of nesting level, also implies global if the name is declared global in the parent block. Unless I've wholly lost track of your intent, which is quite possible.

=== no new content below ===

- implicit class closure, where class lives (parentlocal -> nonlocal in current scope, classlocal in class scope)

Most notably, in the synthetic functions created for generator expressions and comprehensions, a parentlocal declaration in a child scope would imply a parentlocal declaration in the synthetic function as well, propagating back up the chain of nested lexical scopes until it terminated in one of the above three permitted targets. Using the explicit forms would then look like: _from future import parentscopes # Enable the explicit declaration forms class C: classlocal n # Declares n as a cell akin to class rather than a class attribute n = [] @staticmethod def getcount(): return len(n) assert not hasattr(C, "n") assert C.getcount() == 0 def writestoparentscope(): parentlocal outername outername = 42 assert outername == 42 I'm still doubtful the complexity of actually doing that is warranted, but I'm now satisfied the semantics can be well specified in a way that allows us to retain the explanation of generator expressions and comprehensions in terms of their statement level counterparts (with the added bonus of making "class" a little less of a magically unique snowflake along the way).

More information about the Python-Dev mailing list