[Python-Dev] Free threading (original) (raw)

Guido van Rossum guido@python.org
Wed, 08 Aug 2001 00:46:45 -0400

[Guido] > ... > Anyway, I don't see the point: if you want two 100% independent > interpreters, you can get that easily by forking two separate > processes. That's what separate processes are for...

So what is it that multiple interpreters within a single process are for? I'm comfortable with threads, and comfortable with processes, but multiple interpreters not-really-sharing yet not-really-isolated seem mostly an opportunity to track down subtle bugs .

It was always feature number one requested by folks coming from Tcl, where creating multiple interpreters (threaded or not) is an acceptable pastime for people of good manners. But it seems to be wasted on the average Pythoneer.

In Python, there's one good use I can think of: an application may embed Python and occasionally want to run a good chunk of Python code. If it needs each chunk of Python code to run completely independent from each other chunk, creating a new interpreter for each chunk is a good way to avoid that changes to e.g. sys.path or tabnanny's globals made by one chunk affects the next chunk. The hosting application might want the execution to share memory with it, so these interpreters have to live in the same process.

This works well if the next chunk is only started after the previous chunk is done; but then using Py_Initialize() and Py_Finalize() is enough (creating one interpreter at a time). If the chunks need to overlap in time, creating multiple interpreters is the right solution.

But apparently this habit has fallen in disuse -- from looking at the code I believe there's a serious problem with exceptions, where initializing the second interpreter messes up the identities of the exception classes stored in global variables by the first interpreter, and nobody has ever complained about this. Or maybe they are still using Python 1.5.2, which may not have the same problem. Or maybe this is the reason why PyApache has a reptabion of instability. :(

--Guido van Rossum (home page: http://www.python.org/~guido/)