[Python-Dev] Questions about signal handling. (original) (raw)

Eric Snow ericsnowcurrently at gmail.com
Mon Sep 24 16:19:38 EDT 2018

On Mon, Sep 24, 2018 at 11:14 AM Yury Selivanov <yselivanov.ml at gmail.com> wrote:

On Fri, Sep 21, 2018 at 7:04 PM Eric Snow <ericsnowcurrently at gmail.com> wrote: > 1. Why do we restrict calls to signal.signal() to the main thread? > 2. Why must signal handlers run in the main thread? > 3. Why does signal handling operate via the "pending calls" machinery > and not distinctly?

Here's my take on this: Handling signals in a multi-threaded program is hard. Some signals can be delivered to an arbitrary thread, some to the one that caused them. Posix provides lots of mechanisms to tune how signals are received (or blocked) by individual threads, but (a) Python doesn't expose those APIs, (b) using those APIs correctly is insanely hard. By restricting that we can only receive signals in the main thread we remove all that complexity.

Just to be clear, I'm not suggesting that we allow folks to specify in which Python (or kernel) thread a Python-level signal handler is called.

The reason I've asked about signals is because of the semantics under subinterpreters (where there is no "main" thread). However, I don't have any plans to introduce per-interpreter signal handlers. Mostly I want to understand about the "main" thread restriction for the possible impact on subinterpreters.

FWIW, I'm also mildly curious about the value of the "main" thread restriction currently. From what I can tell the restriction was made early on and there are hints in the C code that it's no longer needed. I suspect we still have the restriction solely because no one has bothered to change it. However, I wasn't sure so I figured I'd ask. :)

Restricting that signal.signal() can only be called from the main thread just makes this API more consistent

Yeah, that's what I thought.

(and also IIRC avoids weird sigaction() behaviour when it is called from different threads within one program).

Is there a good place where this weirdness is documented?

Next, you can only call reentrant functions in your signal handlers. For instance, printf() function isn't safe to use. Therefore one common practice is to set a flag that a signal was received and check it later (exactly what we do with the pending calls machinery).

We don't actually use the pending calls machinery for signals though. The only thing we do is always call PyErr_CheckSignals() before making any pending calls. Wouldn't it be equivalent if we called PyErr_CheckSignals() at the beginning of the eval loop right before calling Py_MakePendingCalls()?

This matters to me because I'd like to use "pending" calls for subinterpreters, which means dealing with signals in Py_MakePendingCalls() is problematic. Pulling the PyErr_CheckSignals() call out would eliminate that problem.

Therefore, IMO, the current way we handle signals in Python is the safest, most predictable, and most cross-platform option there is. And changing how Python signals API works with threads in any way will actually break the world.

I agree that the way we deal with signals (i.e. set a flag that is later handled in PyErr_CheckSignals(), protected by the GIL) shouldn't change. My original 3 questions do not relate to that. Looks like that wasn't terribly clear. :)

-eric

More information about the Python-Dev mailing list