[Python-Dev] Making python C-API thread safe (try 2) (original) (raw)

Phillip J. Eby pje at telecommunity.com
Thu Sep 11 14:10:06 EDT 2003

At 07:53 PM 9/11/03 +0300, Harri Pesonen wrote:

Sorry about the previous post! :-)

Background: I'm new to Python. I just embedded Python into a multi-threaded application, and extended Python by callbacks as well. During this process I realised that Python is not internally as beautiful as externally. It is not thread safe. This surprised me, because Python is a quite young programming language. I started thinking how to fix this, how to make Python thread safe. Currently Python does not support multiprocessor machines or hyperthreading processors. It is only a matter of time when this has to be fixed, better soon than later. How to make C-API thread safe: It's quite simple, in fact. Thread should first call PyThreadState* PyNewInterpreter() and use this thread state pointer in every following API call. This thread state pointer should point to a memory structure, that holds everything that Python interpreter needs. There should be no global variables, not even None, except

This only makes individual Python interpreters thread-safe. That might certainly be useful for some embedded applications, but it does nothing for thread-safety within an interpreter.

In other words, Python would still need a global interpreter lock, in order for threaded programs written in Python to behave properly.

So, your solution would help only creators of multithreaded embedded applications, wherein each thread desired an isolated Python interpreter. Meanwhile, it would impose both a one-time and ongoing cost penalty on all other Python users, without giving them any benefit in return.

More information about the Python-Dev mailing list