[Python-Dev] Signals, threads, blocking C functions (original) (raw)

Gustavo Carneiro gjcarneiro at gmail.com
Mon Sep 4 16:52:36 CEST 2006

On 9/4/06, Nick Maclaren <nmm1 at cus.cam.ac.uk> wrote:

"Gustavo Carneiro" <gjcarneiro at gmail.com> wrote: > I am now thinking of something along these lines: > typedef void (*PyPendingCallNotify)(void *userdata); > PyAPIFUNC(void) PyAddPendingCallNotify(PyPendingCallNotify callback, > void *userdata); > PyAPIFUNC(void) PyRemovePendingCallNotify(PyPendingCallNotify > callback, void *userdata);

Why would that help? The problems are semantic, not syntactic. Anthony Baxter isn't exaggerating the problem, despite what you may think from his posting.

You guys are tough customers to please. I am just trying to solve a problem here, not create a new one; you have to believe me.

OK, let's review what we know about current python, signals, and threads:

 1. Python launches threads without touching sigprocmask;
 2. Python installs signal handlers for all signals;
 3. Signals can be delivered to any thread, let's assume (because

of point #1 and not others not mentioned) that we have no control over which threads receive which signals, might as well be random for all we know; 4. Python signal handlers do almost nothing: just sets a flag, and calls Py_AddPendingCall, to postpone the job of handling a signal until a "safer" time. 5. The function Py_MakePendingCalls() should eventually get called at a "safer" time by user or python code. 6. It follows that until Py_MakePendingCalls() is called, the signal will not be handled at all!

Now, back to explaining the problem.

 1. In PyGTK we have a gobject.MainLoop.run() method, which blocks

essentially forever in a poll() system call, and only wakes if/when it has to process timeout or IO event; 2. When we only have one thread, we can guarantee that e.g. SIGINT will always be caught by the thread running the g_main_loop_run(), so we know poll() will be interrupted and a EINTR will be generated, giving us control temporarily back to check for python signals; 3. When we have multiple thread, we cannot make this assumption, so instead we install a timeout to periodically check for signals.

We want to get rid of timeouts. Now my idea: add a Python API to say: "dear Python, please call me when you start having pending calls, even if from a signal handler context, ok?"

From that point on, signals will get handled by Python, python calls PyGTK, PyGTK calls a special API to safely wake up the main loop even from a thread or signal handler, then main loop checks for signal by calling PyErr_CheckSignals(), it is handled by Python, and the process lives happily ever after, or die trying.

I sincerely hope my explanation was satisfactory this time.

Best regards.

PS: there's a "funny" comment in Py_AddPendingCall that suggests it is not very safe against reentrancy problems:

/* XXX Begin critical section */
/* XXX If you want this to be safe against nested
   XXX asynchronous calls, you'll have to work harder! */

Are signal handlers guaranteed to not be interrupted by another signal, at least? What about threads?

More information about the Python-Dev mailing list