[Python-Dev] PEP 567 -- Context Variables (original) (raw)

Yury Selivanov yselivanov.ml at gmail.com
Sun Dec 17 14:49:26 EST 2017

Hi Ben,

On Sun, Dec 17, 2017 at 10:38 AM, Ben Darnell <ben at bendarnell.com> wrote:

On Tue, Dec 12, 2017 at 12:34 PM Yury Selivanov <yselivanov.ml at gmail.com> wrote:

Hi, This is a new proposal to implement context storage in Python. It's a successor of PEP 550 and builds on some of its API ideas and datastructures. Contrary to PEP 550 though, this proposal only focuses on adding new APIs and implementing support for it in asyncio. There are no changes to the interpreter or to the behaviour of generator or coroutine objects. I like this proposal. Tornado has a more general implementation of a similar idea (https://github.com/tornadoweb/tornado/blob/branch4.5/tornado/stackcontext.py), but it also tried to solve the problem of exception handling of callback-based code so it had a significant performance cost (to interpose try/except blocks all over the place). Limiting the interface to coroutine-local variables should keep the performance impact minimal.

Thank you, Ben!

Yes, task local API of PEP 567 has no impact on generators/coroutines. Impact on asyncio should be well within 1-2% slowdown, visible only in microbenchmarks (and asyncio will be 3-6% faster in 3.7 at least due to some asyncio.Task C optimizations).

[..]

One caveat based on Tornado's experience with stackcontext: There are times when the automatic propagation of contexts won't do the right thing (for example, a database client with a connection pool may end up hanging on to the context from the request that created the connection instead of picking up a new context for each query).

I can see two scenarios that could lead to that:

  1. The connection pool explicitly captures the context with 'get_context()' at the point where it was created. It later schedules all of its code within the captured context with Context.run().

  2. The connection pool calls ContextVar.get() once and caches it.

Both (1) and (2) are anti-patterns. The documentation of asyncio and contextvars module will explain that users are supposed to simply call ContextVar.get() whenever they need to get a context value (e.g. there's no need to cache/persist it) and that they should not manage the context manually (just trust asyncio to do that for you).

Thank you, Yury

More information about the Python-Dev mailing list