[Python-Dev] Python 2.3 problem with extention-embedding in multi-threaded and multi sub-interpreter environment (original) (raw)

Atul Kshirsagar atul.kshirsagar at firstlogic.com
Fri Mar 19 12:35:17 EST 2004

I am embedding python in my C++ application. I am using Python 2.3.2 with a C++ extention DLL in multi-threaded environment. I am using SWIG-1.3.19 to generate C++ to Python interface.

Now to explain it in details,

  1. Python initialization [Py_Initialize()] and finalization [Py_Finalize()] is done in the main thread.
  2. For each new thread I create a separate sub-interpreter [Py_NewInterpreter()].
  3. Using PyRun_String("import myModule"...) before execution of python script, extention module is imported.
  4. Each thread executes multiple python script using PyEval_EvalCode() using the class objects in my extention DLL.
  5. Each sub-interpreter is destroyed [Py_EndInterpreter()] at the end of that particular thread.

I am observing that; As explained above when multiple threads are running. And as one of these threads finishes, in other running threads I start getting "TypeError: 'NoneType' object is not callable" error on the methods called on class objects in extention module.

The same code works fine with Python 2.2.2.

I have found these links more or less talking about the same problem migrating from 2.2 to 2.3. http://mail.python.org/pipermail/python-dev/2003-September/038237.html http://mail.python.org/pipermail/python-list/2004-February/206851.html http://mail.python.org/pipermail/python-list/2004-January/204040.html

I guess what is happening is global variables are zapped to "NoneType" when one thread finishes and other thread trying to access them through the Python script (step 4.) this error is generated. But it works sometimes when(guess) the running thread is at step 3. and by importing the module the global variables are re-initialized for "Type" information.

I tried using reload(myModule) to solve the problem but that is generating big memory leak every time it is called.

Is this a know issue with 2.3 (interpreter?) ? Or is there a change for 2.3 in the way embedding should be done in a multi-threaded and multi-sub-interpreter environment ?

Anybody for help ? Thanks, Atul

More information about the Python-Dev mailing list