Interrupting long running PyObject_Call inside subinterpreter (original) (raw)

December 23, 2024, 11:39pm 1

I’m working on Python foreign function interface for Elisp.
Python methods are called in separated pthread with PyObject_Call.
I’m creating sub-interpreters to isolate Elisp packages from each other.
I want user to be able to interrupt long running python methods in same manner as we can press control-C in interactive interpreter.

My problem is PyErr_SetInterrupt seem to not work and i have no idea how to do it.
Naive attempt with signals and PyErr_SetInterrupt is in commit naive attempt at C-g implementation · 813gan/emacspy@2eecf89 · GitHub

ZeroIntensity (Peter Bierma) December 24, 2024, 5:16am 2

Subinterpreters don’t support CTRL+C yet, at least when they’re on the main thread: Ctrl-C Does Not Do Anything While A Subinterpreter Is Running in the Main Thread · Issue #113130 · python/cpython · GitHub
(If you try to run it in a seperate thread, you’ll get a crash: Multithreaded subinterpreters can be running during finalization · Issue #126016 · python/cpython · GitHub)

Speculatively, you could install a signal handler and then use PyThreadState_SetAsyncExc to send a KeyboardInterrupt to the interpreter, but I suspect you’ll run into some interesting edge cases.

813gan (813gan) December 24, 2024, 11:23am 3

But GIL is needed for both PyThreadState_SetAsyncExc and PyObject_Call, no?
I’m a bit confused because documentation for PyObject_Call doesnt say anything about GIL but i got crash when i released GIL before call to it.

813gan (813gan) December 24, 2024, 11:29am 4

Also. thanks for your response especially for links to existing issues.
Lack of reaction under Ctrl-C Does Not Do Anything While A Subinterpreter Is Running in the Main Thread · Issue #113130 · python/cpython · GitHub makes me feel like using sub-interpreters was not good idea in first place.

barry-scott (Barry Scott) December 24, 2024, 12:13pm 5

A long time ago there used to be a callback in the ceval code that you could hook into. I was using it for a while. No idea if that hook is still in the code.
Suggest you read the code of the evaluation loop and see if it is still there.

I used the hook to simulate a KeyboardInterrupt to prevent the python code from locking up the app.

813gan (813gan) December 24, 2024, 4:09pm 6

Hi, thanks for response. I don’t want to go beyond stable ABI as is i need to support wide range of python versions and platforms.

barry-scott (Barry Scott) December 24, 2024, 10:05pm 7

The hook was part of the API at the time I was using it I think.