[Python-Dev] PEP 550 v4 (original) (raw)

Eric Snow ericsnowcurrently at gmail.com
Mon Aug 28 18:14:40 EDT 2017

On Sat, Aug 26, 2017 at 3:09 PM, Nathaniel Smith <njs at pobox.com> wrote:

You might be interested in these notes I wrote to motivate why we need a chain of namespaces, and why simple "async task locals" aren't sufficient:

https://github.com/njsmith/pep-550-notes/blob/master/dynamic-scope.ipynb

Thanks, Nathaniel! That helped me understand the rationale, though I'm still unconvinced chained lookup is necessary for the stated goal of the PEP.

(The rest of my reply is not specific to Nathaniel.)

tl;dr Please:

If I understood right, the problem is that we always want context vars resolved relative to the current frame and then to the caller's frame (and on up the call stack). For generators, "caller" means the frame that resumed the generator. Since we don't know what frame will resume the generator beforehand, we can't simply copy the current LC when a generator is created and bind it to the generator's frame.

However, I'm still not convinced that's the semantics we need. The key statement is "and then to the caller's frame (and on up the call stack)", i.e. chained lookup. On the linked page Nathaniel explained the position (quite clearly, thank you) using sys.exc_info() as an example of async-local state. I posit that that example isn't particularly representative of what we actually need. Isn't the point of the PEP to provide an async-safe alternative to threading.local()?

Any existing code using threading.local() would not expect any kind of chained lookup since threads don't have any. So introducing chained lookup in the PEP is unnecessary and consequently not ideal since it introduces significant complexity.

As the PEP is currently written, chained lookup is a key part of the proposal, though it does not explicitly express this. I suppose this is where my confusion has been.

At this point I think I understand one rationale for the chained lookup functionality; it takes advantage of the cooperative scheduling characteristics of generators, et al. Unlike with threads, a programmer can know the context under which a generator will be resumed. Thus it may be useful to the programmer to allow (or expect) the resumed generator to fall back to the calling context. However, given the extra complexity involved, is there enough evidence that such capability is sufficiently useful? Could chained lookup be addressed separately (in another PEP)?

Also, wouldn't it be equally useful to support chained lookup for function calls? Programmers have the same level of knowledge about the context stack with function calls as with generators. I would expect evidence in favor of chained lookups for generators to also favor the same for normal function calls.

-eric

More information about the Python-Dev mailing list