[Python-Dev] PEP 563: Postponed Evaluation of Annotations (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Thu Nov 9 02:57:39 EST 2017

TL;DR version: I'm now +1 on a string-based PEP 563, with one relatively small quibble regarding the future flag's name.

Putting that quibble first: could we adjust the feature flag to be either "from future import lazy_annotations" or "from future import str_annotations"?

Every time I see "from future import annotations" I think "But we've had annotations since 3.0, why would they need a future import?".

Adding the "lazy_" or "str_" prefix makes the feature flag self-documenting: it isn't the annotations support that's new, it's the fact the interpreter will avoid evaluating them at runtime by treating them as implicitly quoted strings at compile time.

See inline comments for clarifications on what I was attempting to propose in relation to thunks, and more details on why I changed my mind :)

On 9 November 2017 at 14:16, Guido van Rossum <guido at python.org> wrote:

On Wed, Nov 8, 2017 at 5:49 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

On 8 November 2017 at 16:24, Guido van Rossum <guido at python.org> wrote: > I also don't like the idea that there's nothing you can do with a thunk > besides calling it -- you can't meaningfully introspect it (not without > building your own bytecode interpreter anyway). Wait, that wasn't what I was suggesting at all - with thunks exposing their code object the same way a function does (i.e. as a _code_ attribute), the introspection functions in dis would still work on them, so you'd be able to look at things like which variable names they referenced, thus granting the caller complete control over how they resolved those variable names (by setting them in the local namespace passed to the call). I understood that they would be translated to lambda: <expr>. It seems you have a slightly more complex idea but if you're suggesting introspection through dis, that's too complicated for my taste.

Substituting in a lambda expression wouldn't work for the reasons you gave when you objected to that idea (there wouldn't be any way for typing.get_type_hints() to inject "vars(cls)" when evaluating the annotations for method definitions, and enabling a cell-based alternative would be a really intrusive change).

This is why they'd have interesting potential future use cases as general purpose callbacks - every local, nonlocal, global, and builtin name reference would implicitly be an optional parameter (or a required parameter if the name couldn't be resolved as a nonlocal, global, or builtin). Yeah, but that's scope creep for PEP 563. Łukasz and I are interested in gradually restricting the use of annotations to static typing with an optional runtime component. We're not interested in adding different use cases. (We're committed to backwards compatibility, but only until 4.0, with a clear deprecation path.)

Sorry, that was ambiguous wording on my part: the "potential future use cases" there related to thunks in general, not their use for annotations in particular.

APIs like pandas.query are a more meaningful example of where thunks are potentially useful (and that's a problem I've been intermittently pondering since Fernando Perez explained it to me at SciPy a few years back - strings are an OK'ish workaround, but losing syntax highlighting, precompiled code object caching, and other benefits of real Python expressions means they are a workaround).

Instead, thunks would offer all the same introspection features as lambda expressions do, they'd just differ in the following ways:

* the parameter list on their code objects would always be empty * the parameter list for their call method would always be "ns=None" * they'd be compiled without COOPTIMIZED (the same as a class namespace) * they'd look up their closure references using LOADCLASSDEREF (the same as a class namespace) I don't understand the call with "ns-None" thing but I don't expect it matters.

It was an attempted shorthand for the way thunks could handle the method annotations use case in a way that regular lambda expressions can't: "thunk(vars(cls))" would be roughly equivalent to "exec(thunk.code, thunk.globals, vars(cls))", similar to the way class body evaluations works like "exec(body.code, body.globals, mcl.prepare())"

That doesn't make a difference to your decision in relation to PEP 563, though.

That leaves the door open to a future PEP that proposes thunk-based annotations as part of proposing thunks as a new low level delayed evaluation primitive. Sorry, that's not a door I'd like to leave open.

At this point, I'd expect any successful PEP for the thunks idea to offer far more compelling use cases than type annotations - the key detail for me is that even if PEP 563 says "Lazy evaluation as strings means that type annotations do not support lexical closures", injecting attributes into class namespaces will still offer a way for devs to emulate closure references if they really want them.

It's also the case that not supporting lexical closures would likely be better for potential thunk use cases like pandas.query, as in those kinds of use cases, the desired name resolution sequence is "table columns, module globals, builtins" - picking up random function locals as a closure reference and hence keeping them alive indefinitely isn't actually desirable.

So I've reached a point where I'm happy that name resolution in simple string-based delayed evaluation in annotations can be mapped to existing name resolution concepts ("it's like a nested class body, just without lexical closure support, and with method definitions receiving their defining class namespace as a read-only locals()"), and that it won't prevent us from considering potential future enhancements to our syntactic support for ad hoc query APIs.

Those are the two major things that were worrying me about the current draft, so with those concerns resolved, +1 from me.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list