[Python-Dev] PEP 446 (make FD non inheritable) ready for a final review (original) (raw)

Charles-François Natali cf.natali at gmail.com
Fri Aug 23 09:48:15 CEST 2013

Hello,

A couple remarks:

The following functions are modified to make newly created file descriptors non-inheritable by default: [...] os.dup()

then

os.dup2() has a new optional inheritable parameter: os.dup2(fd, fd2, inheritable=True). fd2 is created inheritable by default, but non-inheritable if inheritable is False.

Why does dup2() create inheritable FD, and not dup()?

I think a hint is given a little later:

Applications using the subprocess module with the passfds parameter or using os.dup2() to redirect standard streams should not be affected.

But that's overly-optimistic.

For example, a lot of code uses the guarantee that dup()/open()... returns the lowest numbered file descriptor available, so code like this:

r, w = os.pipe() if os.fork() == 0: # child os.close(r) os.close(1) dup(w)

will break

And that's a lot of code (e.g. that's what _posixsubprocess.c uses, but since it's implemented in C it's wouldn't be affected).

We've already had this discussion, and I stand by my claim that changing the default will break user code. Furthermore, many people use Python for system programming, and this change would be highly surprising.

So no matter what the final decision on this PEP is, it must be kept in mind.

The programming languages Go, Perl and Ruby make newly created file descriptors non-inheritable by default: since Go 1.0 (2009), Perl 1.0 (1987) and Ruby 2.0 (2013).

OK, but do they expose OS file descriptors? I'm sure such a change would be fine for Java, which doesn't expose FDs and fork(), but Python's another story.

Last time, I said that to me, the FD inheritance issue is solved on POSIX by the subprocess module which passes close_fds. In my own code, I use subprocess, which is the "official", portable and safe way to create child processes in Python. Someone using fork() + exec() should know what he's doing, and be able to deal with the consequences: I'm not only talking about FD inheritance, but also about async-signal/multi-threaded safety ;-)

As for Windows, since it doesn't have fork(), it would make sense to make its FD non heritable by default. And then use what you describe here to selectively inherit FDs (i.e. implement keep_fds): """ Since Windows Vista, CreateProcess() supports an extension of the STARTUPINFO struture: the STARTUPINFOEX structure. Using this new structure, it is possible to specify a list of handles to inherit: PROC_THREAD_ATTRIBUTE_HANDLE_LIST. Read Programmatically controlling which handles are inherited by new processes in Win32 (Raymond Chen, Dec 2011) for more information. """

cf

More information about the Python-Dev mailing list