Message 217596 - Python tracker (original) (raw)

It looks like a bug in the subprocess module e.g., if child process does:

sys.stdout.write(sys.stdin.readline()) sys.stdout.flush()

and the parent:

p.stdin.write(line) #NOTE: no flush line = p.stdout.readline()

then a deadlock may happen with bufsize=1 (because it is equivalent to bufsize=-1 in the current code)

Surprisingly, it works if universal_newlines=True but only for C implementation of io i.e., if C extension is disabled:

import io, _pyio io.TextIOWrapper = _pyio.TextIOWrapper

then it blocks forever even if universal_newlines=True with bufsize=1 that is clearly wrong (<-- bug) e.g., test_universal_newlines deadlocks with _pyio.TextIOWrapper

C implementation works because it sets needflush flag even if only write_through is provided 1:

if (self->write_through)
    needflush = 1;
else if (self->line_buffering &&
    (haslf ||
     PyUnicode_FindChar(text, '\r', 0, PyUnicode_GET_LENGTH(text), 1) != -1))
    needflush = 1;

Python io implementation doesn't flush with only write_through flag.

It doesn't deadlock if bufsize=0 whether universal_newlines=True or not.

Note: Python 2.7 doesn't deadlock with bufsize=0 and bufsize=1 in this case as expected.

What is not clear is whether it should work with universal_newline=False and bufsize=1: both current docs and Python 2.7 behaviour say that there should not be a deadlock.

I've updated the docs to mention that bufsize=1 works only with universal_newlines=True and added corresponding tests. I've also updated the subprocess' code to pass line_buffering explicitly.

Patch is uploaded.