[Python-Dev] fork or exec? (original) (raw)

Ben Leslie benno at benno.id.au
Fri Jan 11 08:25:44 CET 2013

On Fri, Jan 11, 2013 at 5:30 PM, Charles-François Natali < cf.natali at gmail.com> wrote:

> That could always be overcome by passing closefds=False explicitly to > subprocess from my code, though, right? I'm not doing that now, but then I'm not > using the esoteric options in python-gnupg code, either.

You could do that, or better explicitly support this option, and only specify this file descriptor in subprocess.Popen keepfds argument. > My point was that the GnuPG usage looked like an example where fds other than > 0, 1 and 2 might be used by design in not-uncommonly-used programs. From a > discussion I had with Barry Warsaw a while ago, I seem to remember that there > was other software which relied on these features. See [1] for details. Yes, it might be used. But I maintain that it should be really rare, and even if it's not, since the "official" way to launch subprocesses is through the subprocess module, FDs > 2 are already closed by default since Python 3.2. And the failure will be immediate and obvious (EBADF).

I suppose I should chime in. I think the pattern of "open some file descriptors, fork, exec" and pass the file descriptors to a child process it a very useful thing, and certainly something I use in my code. Although to be honest, it is usually a C program doing it.

I do this for two reason:

  1. I can have a relatively small program that runs at a rather high privilege level (e.g: root), which acquires protected resources (e.g: binds to port 80, opens some key files that only a restricted user has access to), and then drops privileges, forks (or just execs) and passes those resources to a larger, less trusted program.

  2. I have a monitoring program that opens a socket, and then passes the socket to a child process (via fork, exec). If that program crashes for some reason, the monitoring program can respawn a new child and pass it the exact same socket.

Now I'm not saying these are necessarily common use cases, but I think changing the default behaviour of file-descriptors to be close on exec would violate principle of least surprise for anyone familiar with and expecting UNIX semantics. If you are using fork/exec directly then I think you want these semantics. That being said, I have no problem with subprocess closing file-descriptors before exec()-ing as I don't think a developer would necessarily expect the UNIX semantics on that interface.

Python is not UNIX, but I think if you are directly using the POSIX interfaces they should work (more or less) the same way the would if you were writing a C program. (Some of us still use Python to prototype things that will later be converted to C!).

Cheers,

Benno -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20130111/620cd86a/attachment.html>

More information about the Python-Dev mailing list