[Python-Dev] Possible GIL/threading issue involving subprocess and PyMem_MALLOC... (original) (raw)

Trent Nelson trent at snakebite.org
Fri Dec 21 04:12:31 CET 2012

On Thu, Dec 20, 2012 at 05:47:40PM -0800, Gregory P. Smith wrote:

On Thu, Dec 20, 2012 at 10:43 AM, Trent Nelson <trent at snakebite.org> wrote:

This seems odd to me so I wanted to see what others think. The unit test Lib/unittest/test/testrunner.py:TestTextRunner.testwarnings will eventually hit subprocess.Popen.communicate. The mswindows implementation of this method relies on threads to buffer stdin/stdout. That'll eventually result in PyOsStdioReadline being called without the GIL being held. PyOsStdioReadline calls PyMemMALLOC, PyMemFREE and possibly PyMemREALLOC. Those threads are implemented in Python so how would the GIL ever not be held? -gps

PyOS_Readline drops the GIL prior to calling PyOS_StdioReadline:

    Py_BEGIN_ALLOW_THREADS

--------^^^^^^^^^^^^^^^^^^^^^^ #ifdef WITH_THREAD PyThread_acquire_lock(_PyOS_ReadlineLock, 1); #endif

    /* This is needed to handle the unlikely case that the
     * interpreter is in interactive mode *and* stdin/out are not
     * a tty.  This can happen, for example if python is run like
     * this: python -i < test1.py
     */
    if (!isatty (fileno (sys_stdin)) || !isatty (fileno (sys_stdout)))
        rv = PyOS_StdioReadline (sys_stdin, sys_stdout, prompt);

-----------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ else rv = (*PyOS_ReadlineFunctionPointer)(sys_stdin, sys_stdout, prompt); Py_END_ALLOW_THREADS

    Trent.

More information about the Python-Dev mailing list