[Python-Dev] GIL vs thread state (original) (raw)

Thomas Heller theller@python.net
14 Apr 2003 16:58:50 +0200

Guido van Rossum <guido@python.org> writes:

> The docs for PyThreadStateClear() state that the interpreter lock must > be held. > > I had this code in ctypes to delete the thread state and release the lock: > > static void LeavePython(char *msg) > { > PyThreadState *pts = PyThreadStateSwap(NULL); > if (!pts) > PyFatalError("wincall (LeavePython): ThreadState is NULL?"); > PyThreadStateClear(pts); > PyThreadStateDelete(pts); > PyEvalReleaseLock(); > } > > and (under certain coditions, when ptr->frame was not NULL), got

What is ptr->frame? A typo for pts->frame?

Right, sorry.

If pts->frame is not NULL, I'd expect a warning from PyThreadStateClear(): "PyThreadStateClear: warning: thread still has a frame\n". You mean this code, from Python/pystate.h?

void PyThreadState_Clear(PyThreadState *tstate) { if (Py_VerboseFlag && tstate->frame != NULL) fprintf(stderr, "PyThreadState_Clear: warning: thread still has a frame\n");

ZAP(tstate->frame);

ZAP(tstate->dict);

... }

Py_VerboseFlag is 0 set in my case, so no warning is printed.

> "Fatal Python error: PyThreadStateGet: no current thread" in the call > to PyThreadStateClear(). That's strange, because I cannot trace the code in there to such a call. (Unless it is in a destructor.

It is in a destructor: frame_dealloc, called from ZAP(tstate->frame).

Can you tell more about where the PyThreadStateGet() call was?)

This function allocates the threadstate for me:

static void EnterPython(char *msg) { PyThreadState *pts; PyEval_AcquireLock(); pts = PyThreadState_New(g_interp); if (!pts) Py_FatalError("wincall: Could not allocate ThreadState"); if (NULL != PyThreadState_Swap(pts)) Py_FatalError("wincall (EnterPython): thread state not == NULL?"); }

To explain the picture a little better, here is the sequence of calls:

Python calls into a C extension. The C extension does

Py_BEGIN_ALLOW_THREADS call_a_C_function() Py_END_ALLOW_THREADS

The call_a_C_function calls back into C code like this:

void MyCallback(void) { EnterPython(); /* acquire the lock, and create a thread state / execute_some_python_code(); LeavePython(); / destroy the thread state, and release the lock */ }

Now, the execute_some_python_code() section is enclosed in a win32 structured exception handling block, and it may return still with a frame in the threadstate, as it seems.

Oops, I just tried the code in CVS python, and the problem goes away. Same for 2.3a2. But my code has to run in 2.2.2 as well...

Thomas

Here's the stack from python 2.2.2:

NTDLL! 77f6f570() PyThreadState_Get() line 246 + 10 bytes PyErr_Fetch(_object * * 0x0012f944, _object * * 0x0012f954, _object * * 0x0012f948) line 215 + 5 bytes call_finalizer(_object * 0x0095ef20) line 382 + 17 bytes subtype_dealloc(_object * 0x0095ef20) line 434 + 9 bytes _Py_Dealloc(_object * 0x0095ef20) line 1837 + 7 bytes frame_dealloc(_frame * 0x00890c20) line 82 + 79 bytes _Py_Dealloc(_object * 0x00890c20) line 1837 + 7 bytes PyThreadState_Clear(_ts * 0x0095d2a0) line 174 + 86 bytes LeavePython(char * 0x1001125c) line 41 + 10 bytes