[Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446) (original) (raw)

Guido van Rossum guido at python.org
Wed Jul 24 19:56:05 CEST 2013

On Tue, Jul 23, 2013 at 4:21 PM, Richard Oudkerk <shibturn at gmail.com> wrote:

On 23/07/2013 11:45pm, Victor Stinner wrote:

Said differently: the HANDLEFLAGINHERIT flag only has an effect on handles, as indicated in its name. On Windows, filedescriptors are never inherited (are always closed) in child processes. I don't think that it is possible to inherit file descriptors on Windows. Actually, you can inherit fds if you use os.spawnv() instead of subprocess.Popen().

Wow. Indeed you can -- I just tested this myself. How is this accomplished? I guess the CRT has a backchannel to talk to itself when it creates a process using spawn*?

This is the only reason I can think of for the odd default in the CRT of opening file descriptors inheritable by default, which Victor discovered. (But it doesn't explain why os.pipe() creates uninheritable fds.)

If it weren't for this I would definitely vote to change the default on Windows throughout the stdlib to create file descriptors whose handles aren't inheritable. (Perhaps with a different policy for stdin/stdout/stderr, which seem to be treated specially at the handle level.)

I'm about ready to give up hope that we'll ever have a decent way to deal with this.

But I'm also ready to propose that all this is such a mess that we should change the default fd/handle inheritance to False, across platforms, and damn the torpedoes -- i.e. accept breaking all existing 3rd party UNIX code for subprocess creation that bypasses the subprocess module, as well as breaking uses of os.spawn*() on both platforms that depend on FD inheritance beyond stdin/stdout/stderr).

With the new, sane default, all we need instead of PEP 446 is a way to make an FD inheritable after it's been created, which can be a single os.make_inheritable(fd) call that you must apply to the fileno() of the stream or socket object you want inherited (or directly to a FD you created otherwise, e.g. with os.pipe()). On Windows, this should probably only work with os.spawn*(), since otherwise you need handle inheritance, not FD inheritance, and that's a non-portable concept anyway.

We can fix multiprocessing any anything else in the stdlib that this breaks, I presume.

To reduce the need for 3rd party subprocess creation code, we should have better daemon creation code in the stdlib -- I wrote some damn robust code for this purpose in my previous job, but it never saw the light of day.

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list