[Python-Dev] Questions about signal handling. (original) (raw)

Eric Snow ericsnowcurrently at gmail.com
Tue Sep 25 11:09:26 EDT 2018

On Tue, Sep 25, 2018 at 1:45 AM Victor Stinner <vstinner at redhat.com> wrote:

Please don't rely on this ugly API. By design, PyAddPendingCall() tries 100 times to acquire the lock: if it fails to acquire the lock, it does notthing... your callback is ignored...

Yeah, there are issues with pending calls as implemented. Furthermore, I'm not clear on why it was made a public API in the first place. Ultimately I'd like to deprecate Py_AddPendingCall and Py_MakePendingCalls (but that's not my priority right now). Regardless, the underlying machinery matches what I need for interpreter isolation right now. See below.

By the way, recently, we had to fix yet another bug in signal handling. A new function has been added: [snip]

I saw that. If anything, it's more justification for separating signals from the pending calls machinery. :)

If you want to exchange commands between two interpreters, which I see as two threads, I suggest to use two queues and something to consume queues.

Sure. However, to make that work I'd end up with something that's almost identical to the existing pending calls machinery. Since the function-to-run may call Python code, there must be an active PyThreadState and the GIL (or, eventually, target interpreter's lock) must be held. That is what Py_MakePendingCalls gives us already. Injecting the pending call into the eval loop (as we do today) means we don't have to create a new thread just for handling pending calls. [1]

In the short term I'd like to stick with existing functionality as much as possible. Doing so has a number of benefits. That's been one of my guiding principles as I've worked toward the multi-core Python goal. That said, I agree that the pending calls machinery has some deficiencies that should be fixed or something better should replace it. I just don't want that to get in the way of short-term goals, especially as I have limited time for this.

-eric

[1] FWIW, a separate thread for "asynchronous" operations (ones that interrupt the eval loop, e.g. "pending" calls and signals) might be a viable approach now that we require platforms to support threading. The way we interrupt the eval loop currently seems more complex (and inefficient) than necessary. I was thinking about this last week and plan to explore it further at some point in the future. For now, though, I'd like to keep the focus on more immediate needs. :)

More information about the Python-Dev mailing list