select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module)) (original) (raw)

Chris McDonough chrism at plope.com
Wed Aug 4 18:27:18 CEST 2004

On Wed, 2004-08-04 at 11:20, Guido van Rossum wrote:

> Doesn't select() effectively busy-wait on "real" files (pipes and file > descriptors obtained via open(), as opposed to network sockets) on most > (all?) UNIXen? At least this has been my finding under Linux.

select() doesn't make sense for regular files, so tail -f can't use it. For Unix pipes, it works as advertised. On Windows, it only applies to sockets.

Aha. I think I understand now.

Sorry for going off-topic, but this program (UNIX-only):

import select import errno import fcntl import os

def go(r, timeout=1): while 1: try: r, w, x = select.select(r, [], [], timeout) print r except select.error, err: if err[0] != errno.EINTR: raise r = w = x = []

p_in, p_out = os.pipe()

for fd in p_in, p_out: flags = fcntl.fcntl(fd, fcntl.F_GETFL) fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NDELAY)

os.close(p_in)

go([p_out])

... will busywait in select (p_out is always in the ready state; the select timeout is never reached).

But if you comment out the "os.close(p_in)" line, it no longer busywaits (the select timeout is reached on every iteration). At least this is the behavior under Linux.

This is a little unfortunate because the normal dance when communicating between parent and child in order to capture the output of a child process seems to be:

  1. In the parent process, create a set of pipes that will represent stdin/stdout/stderr of the child.

  2. fork

  3. In the parent process, close the "unused" fds representing the ends of the pipes that the child will use as fds for stdin/stderr/stdout.

  4. In the child process, dup these same fds to stdin, stdout, and stderr.

... so select seems to be unuseful here, unless step 3 isn't actually necessary.

Still a bit confused and researching...

More information about the Python-Dev mailing list