[Python-Dev] PEP 550 v3 (original) (raw)

Guido van Rossum guido at python.org
Mon Aug 21 01:03:20 EDT 2017

On Sun, Aug 20, 2017, 03:08 Antoine Pitrou <solipsis at pitrou.net> wrote:

On Sat, 19 Aug 2017 17:21:03 -0700 Guido van Rossum <gvanrossum at gmail.com> wrote: > The way we came to "logical context" was via "logical thread (of control)", > which is distinct from OS thread. But I think we might need to search for > another term...

Perhaps "task context"? A "task" might be a logical thread, OS thread, or anything else that deserves a distinct set of implicit parameters. I think you're on to something here, though I hesitate to use "task" because asyncio.Task is a specific implementation of it.

On Sun, Aug 20, 2017 at 7:04 PM, Brett Cannon <brett at python.org> wrote:

Maybe this is skirting too loose to the dynamic scoping, but maybe ContextFrame? This does start to line up with frames of execution which I know is a bit low-level, but then again most people will never need to know about this corner of Python.

I've been thinking that the missing link here may be the execution stack. A logical thread (LT) has a "logical stack" (LS). While a thread of control is a fairly fuzzy concept (once you include things that aren't OS threads), an execution stack is a concrete object, even if not all logical threads represent their execution stack in the same way. For example, a suspended asyncio Task has a stack that is represented by a series of stack frames linked together by await (or yield from), and greenlet apparently uses a different representation again (their term is micro-thread -- maybe we could also do something with that?).

Here's the result of some more thinking about this PEP that I've been doing while writing and rewriting this message (with a surprising ending).

Let's say that the details of scheduling an LT and managing its mapping onto an LS is defined by a "framework". In this terminology, OS threads are a framework, as are generators, asyncio, and greenlet. There are potentially many different such frameworks. (Some others include Twisted, Tornado and concurrent.futures.ThreadPoolExecutor.)

The PEP's big idea is to recognize that there are also many different, non-framework, libraries (e.g. Decimal or Flask) that need to associate some data with an LT. The PEP therefore proposes APIs that allow libraries to do this without caring about what framework is managing the LT, and vice versa (the framework doesn't have to care about how libraries use the per-LT data).

The proposed APIs uses two sets of concepts: one set for the framework and one for the library.

The library-facing API is simple: create a ContextKey (CK) instance as a global variable in the library, and use its get() and set() methods to access and manipulate the data for that library associated with the current logical thread (LT). Its role is similar to threading.local(), although the API and implementation are completely different, and threading.local() is tied to a specific framework (OS threads).

For frameworks the API is more complicated. There are two new classes, LogicalContext (LC) and ExecutionContext (EC). The PEP gives pseudo code suggesting that LC is/contains a dict (whose items are (CK, value) pairs) and an EC is/contains a list of LCs. But in actuality that's only one possible implementation (and not the one proposed for CPython). The key idea is rather that a framework needs to be able to take the data associated with one LT and clone it as the starting point for the data associated for a new LT. This cloning is done by sys.get_execution_context(), and the PEP proposes to use a Hash Array Mapped Trie (HAMT) as the basis for the implementation of LC and EC, to make this cloning fast. IIUC it needs to be fast to match the speed with which many frameworks create and destroy their LTs.

The PEP proposes a bunch of new functions in sys for frameworks to manipulate LCs and ECs and their association with the current OS-level thread. Note that OS threads are important here because in the end all frameworks build on top of them.

Honestly I'm not sure we need the distinction between LC and EC. If you read carefully some of the given example code seems to confuse them. If we could get away with only a single framework-facing concept, I would be happy calling it ExecutionContext.

(Another critique of the proposal I have is that it adds too many similarly-named functions to sys. But this email is already too long and I need to go to bed.)

-- --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20170820/24d6cf82/attachment.html>

More information about the Python-Dev mailing list