[Python-Dev] Questions about signal handling. (original) (raw)

Gregory P. Smith greg at krypto.org
Tue Sep 25 11:30:03 EDT 2018

On Mon, Sep 24, 2018 at 1:20 PM Eric Snow <ericsnowcurrently at gmail.com> wrote:

On Mon, Sep 24, 2018 at 11:14 AM Yury Selivanov <yselivanov.ml at gmail.com> wrote: > On Fri, Sep 21, 2018 at 7:04 PM Eric Snow <ericsnowcurrently at gmail.com> wrote: > > 1. Why do we restrict calls to signal.signal() to the main thread? > > 2. Why must signal handlers run in the main thread? > > 3. Why does signal handling operate via the "pending calls" machinery > > and not distinctly? > > Here's my take on this: > > Handling signals in a multi-threaded program is hard. Some signals can > be delivered to an arbitrary thread, some to the one that caused them. > Posix provides lots of mechanisms to tune how signals are received (or > blocked) by individual threads, but (a) Python doesn't expose those > APIs, (b) using those APIs correctly is insanely hard. By restricting > that we can only receive signals in the main thread we remove all that > complexity.

Just to be clear, I'm not suggesting that we allow folks to specify in which Python (or kernel) thread a Python-level signal handler is called. The reason I've asked about signals is because of the semantics under subinterpreters (where there is no "main" thread). However, I don't have any plans to introduce per-interpreter signal handlers. Mostly I want to understand about the "main" thread restriction for the possible impact on subinterpreters. FWIW, I'm also mildly curious about the value of the "main" thread restriction currently. From what I can tell the restriction was made early on and there are hints in the C code that it's no longer needed. I suspect we still have the restriction solely because no one has bothered to change it. However, I wasn't sure so I figured I'd ask. :)

We can't change the API of the main thread being where signal handlers are executed by default.

If a signal handler raised an exception in a daemon thread, the process would not die when it goes uncaught (ie: why KeyboardInterrupt works). The daemon thread ends and the rest of the process is unaware of that. Many existing Python signal handlers expect to only be called from the main thread.

If we wanted to change this, we'd probably want to have users declare which thread(s) are allowed to execute which signal handlers at signal handler registration time and whether they are executed by only one of those threads or by all of those threads. Not semantics I expect most people are used to because I'm not aware of any other language doing that. But I don't see a compelling use case for implementing such complexity. Maybe something like that would make sense for subinterpreter delegation only? I'm not sure. I'd start without signals at all in subinterpreters before making such a decision.

Python keeping signals simple has long been cited as a feature of the VM.

-gps

> Restricting that signal.signal() can only be called from > the main thread just makes this API more consistent Yeah, that's what I thought. > (and also IIRC > avoids weird sigaction() behaviour when it is called from different > threads within one program). Is there a good place where this weirdness is documented? > Next, you can only call reentrant functions in your signal handlers. > For instance, printf() function isn't safe to use. Therefore one > common practice is to set a flag that a signal was received and check > it later (exactly what we do with the pending calls machinery). We don't actually use the pending calls machinery for signals though. The only thing we do is always call PyErrCheckSignals() before making any pending calls. Wouldn't it be equivalent if we called PyErrCheckSignals() at the beginning of the eval loop right before calling PyMakePendingCalls()? This matters to me because I'd like to use "pending" calls for subinterpreters, which means dealing with signals in PyMakePendingCalls() is problematic. Pulling the PyErrCheckSignals() call out would eliminate that problem. > Therefore, IMO, the current way we handle signals in Python is the > safest, most predictable, and most cross-platform option there is. > And changing how Python signals API works with threads in any way will > actually break the world. I agree that the way we deal with signals (i.e. set a flag that is later handled in PyErrCheckSignals(), protected by the GIL) shouldn't change. My original 3 questions do not relate to that. Looks like that wasn't terribly clear. :) -eric

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/greg%40krypto.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20180925/e52a93cd/attachment-0001.html>

More information about the Python-Dev mailing list