[Python-Dev] subprocess.Popen(.... stdout=IGNORE, ...) (original) (raw)

Martin Blais blais at furius.ca
Mon Jun 12 01:59:47 CEST 2006

In the subprocess module, by default the files handles in the child are inherited from the parent. To ignore a child's output, I can use the stdout or stderr options to send the output to a pipe::

p = Popen(command, stdout=PIPE, stderr=PIPE)

However, this is sensitive to the buffer deadlock problem, where for example the buffer for stderr might become full and a deadlock occurs because the child is blocked on writing to stderr and the parent is blocked on reading from stdout or waiting for the child to finish.

For example, using this command will cause deadlock::

call('cat /boot/vmlinuz'.split(), stdout=PIPE, stderr=PIPE)

Popen.communicate() implements a solution using either select() or multiple threads (under Windows) to read from the pipes, and returns the strings as a result. It works out like this::

p = Popen(command, stdout=PIPE, stderr=PIPE) output, errors = p.communicate() if p.returncode != 0: …

Now, as a user of the subprocess module, sometimes I just want to call some child process and simply ignore its output, and to do so I am forced to use communicate() as above and wastefully capture and ignore the strings. This is actually quite a common use case. "Just run something, and check the return code". Right now, in order to do this without polluting the parent's output, you cannot use the call() convenience (or is there another way?).

A workaround that works under UNIX is to do this::

FNULL = open('/dev/null', 'w') returncode = call(command, stdout=FNULL, stderr=FNULL)

Some feedback requested, I'd like to know what you think:

  1. Would it not be nice to add a IGNORE constant to subprocess.py that would do this automatically?, i.e. ::

    returncode = call(command, stdout=IGNORE, stderr=IGNORE)

    Rather than capture and accumulate the output, it would find an appropriate OS-specific way to ignore the output (the /dev/null file above works well under UNIX, how would you do this under Windows? I'm sure we can find something.)

  2. call() should be modified to not be sensitive to the deadlock problem, since its interface provides no way to return the contents of the output. The IGNORE value provides a possible solution for this.

  3. With the /dev/null file solution, the following code actually works without deadlock, because stderr is never blocked on writing to /dev/null::

    p = Popen(command, stdout=PIPE, stderr=IGNORE) text = p.stdout.read() retcode = p.wait()

    Any idea how this idiom could be supported using a more portable solution (i.e. how would I make this idiom under Windows, is there some equivalent to /dev/null)?

More information about the Python-Dev mailing list