[Python-Dev] GIL removal question (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Sat Aug 13 17:43:46 CEST 2011

On Sat, 13 Aug 2011 09:08:16 -0400 Guido van Rossum <guido at python.org> wrote:

And, though mostly off-topic, the worst problem with C code, calling back into Python, and the GIL that I have seen (several times): Suppose you are calling some complex C library that creates threads itself, where those threads may also call back into Python. Here you have to put a block around each Python callback that acquires the GIL before and releases it after, since the new threads (created by C code) start without the GIL acquired. I remember a truly nasty incident where the latter was done, but the main thread did not release the GIL since it was returning directly to Python (which would of course release the GIL every so many opcodes so the callbacks would run). But under certain conditions the block with the acquire-release-GIL code around a Python callback was invoked in the main thread (when a validation problem was detected early), and since the main thread didn't release the GIL around the call into the C code, it hung in a nasty spot. Add many layers of software, and a hard-to-reproduce error condition that triggers this, and you have a problem that's very hard to debug...

These days we have PyGILState_Ensure(): http://docs.python.org/dev/c-api/init.html#PyGILState_Ensure

and even dedicated documentation: http://docs.python.org/dev/c-api/init.html#non-python-created-threads

;)

Regards

Antoine.

More information about the Python-Dev mailing list