[Python-Dev] BEGIN_ALLOW_THREADS (original) (raw)

Martin Sj�gren martin@strakt.com
Wed, 8 Aug 2001 09:14:04 +0200

On Mon, Jul 23, 2001 at 02:21:57PM +0200, Martin Sj=F6gren wrote:

Is there a reason the PyBEGINALLOWTHREADS and PyENDALLOWTHREADS don't allow an argument specifying what variable to save the state to? = I needed this myself so I wrote the following: =20 #ifdef WITHTHREAD _# define MYBEGINALLOWTHREADS(st) _ { st =3D PyEvalSaveThread(); } _# define MYENDALLOWTHREADS(st) _ { PyEvalRestoreThread(st); st =3D NULL; } #else # define MYBEGINALLOWTHREADS(st) # define MYENDALLOWTHREADS(st) { st =3D NULL; } #endif =20 It works just fine but has one drawback: Whenever PyBEGINALLOWTHREAD= S changes, I have to change my macros too. =20 Wouldn't it be reasonable to supply two sets of macros, one that allows exactly this, and one that does what PyBEGINALLOWTHREADS currently does.

If you're wondering why I'm replying to my own age-old-mail, it is becaus= e I noticed the following snippet in python-announce-list:

""" Martin Sj=F6gren wondered why the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros don't take an argument specifying a variable to save the thread state to. He said he needed this, but didn't explain why. """

Looking back I see that, no, I did not motivate that one bit :)

Here's the scoop:

One of the functions (say foo()) in my extension module calls another function (real_foo()) which might block since it does creepy things like I/O. Being a nice fella I surrounded the call with Py_BEGIN_ALLOW_THREAD= S and Py_END_ALLOW_THREADS so that other threads could jump in and do stuff.

The problem is that real_foo() might call a previously defined callback (written in C) which in turn has to call a Python callback (what good is an extension module if you have to write your callbacks in C?), but when = I try to PyEval_CallObject() it - KABOOM! Segfault somewhere in the interpreter (on PyFrame_New() or something like that).

After a lot of thinking (and a lot of help from python-list) I figured ou= t that the state I push away using Py_BEGIN_ALLOW_THREADS must be retrieved again before the callback is called, or PyFrame_New() won't like me. Problem is that that state variable is declared in Py_BEGIN_ALLOW_THREADS and there's no way to reach it (and it also includes a naughty { with the corresponding } in the Py_END_ALLOW_THREADS so it won't work to use them anyway).

My solution: I wrote the two macros above. This works for me, but obviously I have to update my macros when the "real" macros in the Python distribution change.

Okay, that's a more thorough explanation, I hope everybody are happy :)

Martin

--=20 Martin Sj=F6gren martin@strakt.com ICQ : 41245059 Phone: +46 (0)31 405242 Cell: +46 (0)739 169191 GPG key: http://www.strakt.com/~martin/gpg.html