[Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors (original) (raw)

Victor Stinner victor.stinner at gmail.com
Sun Jan 13 01:38:16 CET 2013

HTML version: http://www.python.org/dev/peps/pep-0433/

PEP: 433 Title: Add cloexec argument to functions creating file descriptors Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Victor Stinner <victor.stinner at gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 10-January-2013 Python-Version: 3.4

Abstract

This PEP proposes to add a new optional argument cloexec on functions creating file descriptors in the Python standard library. If the argument is True, the close-on-exec flag will be set on the new file descriptor.

Rationale

On UNIX, subprocess closes file descriptors greater than 2 by default since Python 3.2 [#subprocess_close]_. All file descriptors created by the parent process are automatically closed. xmlrpc.server.SimpleXMLRPCServer sets the close-on-exec flag of the listening socket, the parent class socketserver.BaseServer does not set this flag.

There are other cases creating a subprocess or executing a new program where file descriptors are not closed: functions of the os.spawn*() family and third party modules calling exec() or fork() + exec(). In this case, file descriptors are shared between the parent and the child processes which is usually unexpected and causes various issues.

This PEP proposes to continue the work started with the change in the subprocess, to fix the issue in any code, and not just code using subprocess.

Inherited file descriptors issues

Closing the file descriptor in the parent process does not close the related resource (file, socket, ...) because it is still open in the child process.

The listening socket of TCPServer is not closed on exec(): the child process is able to get connection from new clients; if the parent closes the listening socket and create a new listening socket on the same address, it would get an "address already is used" error.

Not closing file descriptors can lead to resource exhaustion: even if the parent closes all files, creating a new file descriptor may fail with "too many files" because files are still open in the child process.

Security

Leaking file descriptors is a major security vulnerability. An untrusted child process can read sensitive data like passwords and take control of the parent process though leaked file descriptors. It is for example a known vulnerability to escape from a chroot.

Atomicity

Using fcntl() to set the close-on-exec flag is not safe in a multithreaded application. If a thread calls fork() and exec() between the creation of the file descriptor and the call to fcntl(fd, F_SETFD, new_flags): the file descriptor will be inherited by the child process. Modern operating systems offer functions to set the flag during the creation of the file descriptor, which avoids the race condition.

Portability

Python 3.2 added socket.SOCK_CLOEXEC flag, Python 3.3 added os.O_CLOEXEC flag and os.pipe2() function. It is already possible to set atomically close-on-exec flag in Python 3.3 when opening a file and creating a pipe or socket.

The problem is that these flags and functions are not portable: only recent versions of operating systems support them. O_CLOEXEC and SOCK_CLOEXEC flags are ignored by old Linux versions and so FD_CLOEXEC flag must be checked using fcntl(fd, F_GETFD). If the kernel ignores O_CLOEXEC or SOCK_CLOEXEC flag, a call to fcntl(fd, F_SETFD, flags) is required to set close-on-exec flag.

.. note:: OpenBSD older 5.2 does not close the file descriptor with close-on-exec flag set if fork() is used before exec(), but it works correctly if exec() is called without fork().

Scope

Applications still have to close explicitly file descriptors after a fork(). The close-on-exec flag only closes file descriptors after exec(), and so after fork() + exec().

This PEP only change the close-on-exec flag of file descriptors created by the Python standard library, or by modules using the standard library. Third party modules not using the standard library should be modified to conform to this PEP. The new os.set_cloexec() function can be used for example.

Impacted functions:

Impacted modules:

XXX Should subprocess.Popen set the close-on-exec flag on file XXX XXX descriptors of the constructor the pass_fds argument? XXX

.. note:: See Close file descriptors after fork_ for a possible solution for fork() without exec().

Proposal

This PEP proposes to add a new optional argument cloexec on functions creating file descriptors in the Python standard library. If the argument is True, the close-on-exec flag will be set on the new file descriptor.

Add a new function:

Add a new optional cloexec argument to:

The default value of the cloexec argument is False to keep the backward compatibility.

The close-on-exec flag will not be set on file descriptors 0 (stdin), 1 (stdout) and 2 (stderr), because these files are expected to be inherited. It would still be possible to set close-on-exec flag explicitly using os.set_cloexec().

Drawbacks:

Alternatives

Always set close-on-exec flag

Always set close-on-exec flag on new file descriptors created by Python. This alternative just changes the default value of the new cloexec argument.

If a file must be inherited by child processes, cloexec=False argument can be used.

subprocess.Popen constructor has an pass_fds argument to specify which file descriptors must be inherited. The close-on-exec flag of these file descriptors must be changed with os.set_cloexec().

Example of functions creating file descriptors which will be modified to set close-on-exec flag:

Many functions are impacted indirectly by this alternative. Examples:

Advantages of setting close-on-exec flag by default:

Drawbacks of setting close-on-exec flag by default:

Backward compatibility: only a few programs rely on inherance of file descriptors, and they only pass a few file descriptors, usually just one. These programs will fail immediatly with EBADF error, and it will be simple to fix them: add cloexec=False argument or use os.set_cloexec(fd, False).

The subprocess module will be changed anyway to unset close-on-exec flag on file descriptors listed in the pass_fds argument of Popen constructor. So it possible that these programs will not need any fix if they use the subprocess module.

Add a function to set close-on-exec flag by default

An alternative is to add also a function to change globally the default behaviour. It would be possible to set close-on-exec flag for the whole application including all modules and the Python standard library. This alternative is based on the Proposal_ and adds extra changes.

Add new functions:

The major change is that the default value of the cloexec argument is sys.getdefaultcloexec(), instead of False.

When sys.setdefaultcloexec(True) is called to set close-on-exec by default, we have the same drawbacks than Always set close-on-exec flag_ alternative.

There are additionnal drawbacks of having two behaviours depending on sys.getdefaultcloexec() value:

Close file descriptors after fork

This PEP does not fix issues with applications using fork() without exec(). Python needs a generic process to register callbacks which would be called after a fork, see Add an 'afterfork' module_. Such registry could be used to close file descriptors just after a fork().

Drawbacks:

open(): add "e" flag to mode

A new "e" mode would set close-on-exec flag (best-effort).

This alternative only solves the problem for open(). socket.socket() and os.pipe() do not have a mode argument for example.

Since its version 2.7, the GNU libc supports "e" flag for fopen(). It uses O_CLOEXEC if available, or use fcntl(fd, F_SETFD, FD_CLOEXEC). With Visual Studio, fopen() accepts a "N" flag which uses O_NOINHERIT.

Applications using inherance of file descriptors

Most developers don't know that file descriptors are inherited by default. Most programs do not rely on inherance of file descriptors. For example, subprocess.Popen was changed in Python 3.2 to close all file descriptors greater than 2 in the child process by default. No user complained about this behavior change.

Network servers using fork may want to pass the client socket to the child process. For example, on UNIX a CGI server pass the socket client through file descriptors 0 (stdin) and 1 (stdout) using dup2(). This specific case is not impacted by this PEP because the close-on-exec flag is never set on file descriptors smaller than 3.

To access a restricted resource like creating a socket listening on a TCP port lower than 1024 or reading a file containing sensitive data like passwords, a common practice is: start as the root user, create a file descriptor, create a child process, pass the file descriptor to the child process and exit. Security is very important in such use case: leaking another file descriptor would be a critical security vulnerability (see Security_). The root process may not exit but monitors the child process instead, and restarts a new child process and pass the same file descriptor if the previous child process crashed.

Example of programs taking file descriptors from the parent process using a command line option:

On Linux, it is possible to use "/dev/fd/<fd>" filename to pass a file descriptor to a program expecting a filename.

Performances

Setting close-on-exec flag may require additional system calls for each creation of new file descriptors. The number of additional system calls depends on the method used to set the flag:

XXX Benchmark the overhead for these 4 methods. XXX

Implementation

os.set_cloexec(fd, cloexec)

Best-effort by definition. Pseudo-code::

if os.name == 'nt':
    def set_cloexec(fd, cloexec=True):
        SetHandleInformation(fd, HANDLE_FLAG_INHERIT,
                             int(cloexec))
else:
    fnctl = None
    ioctl = None
    try:
        import ioctl
    except ImportError:
        try:
            import fcntl
        except ImportError:
            pass
    if ioctl is not None and hasattr('FIOCLEX', ioctl):
        def set_cloexec(fd, cloexec=True):
            if cloexec:
                ioctl.ioctl(fd, ioctl.FIOCLEX)
            else:
                ioctl.ioctl(fd, ioctl.FIONCLEX)
    elif fnctl is not None:
        def set_cloexec(fd, cloexec=True):
            flags = fcntl.fcntl(fd, fcntl.F_GETFD)
            if cloexec:
                flags |= FD_CLOEXEC
            else:
                flags &= ~FD_CLOEXEC
            fcntl.fcntl(fd, fcntl.F_SETFD, flags)
    else:
        def set_cloexec(fd, cloexec=True):
            raise NotImplementedError(
                "close-on-exec flag is not supported "
                "on your platform")

ioctl is preferred over fcntl because it requires only one syscall, instead of two syscalls for fcntl.

.. note:: fcntl(fd, F_SETFD, flags) only supports one flag (FD_CLOEXEC), so it would be possible to avoid fcntl(fd, F_GETFD). But it may drop other flags in the future, and so it is safer to keep the two functions calls.

.. note:: fopen() function of the GNU libc ignores the error if fcntl(fd, F_SETFD, flags) failed.

open()

os.dup()

os.dup2()

os.pipe()

socket.socket()

socket.socketpair()

socket.socket.accept()

Backward compatibility

There is no backward incompatible change. The default behaviour is unchanged: the close-on-exec flag is not set by default.

Appendix: Operating system support

Windows

Windows has an O_NOINHERIT flag: "Do not inherit in child processes".

For example, it is supported by open() and _pipe().

The value of the flag can be modified using: SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1).

CreateProcess() has an bInheritHandles argument: if it is FALSE, the handles are not inherited. It is used by subprocess.Popen with close_fds option.

fcntl

Functions:

Availability: AIX, Digital UNIX, FreeBSD, HP-UX, IRIX, Linux, Mac OS X, OpenBSD, Solaris, SunOS, Unicos.

ioctl

Functions:

Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD.

Atomic flags

New flags:

On Linux older than 2.6.23, O_CLOEXEC flag is simply ignored. So we have to check that the flag is supported by calling fcntl(). If it does not work, we have to set the flag using fcntl().

XXX what is the behaviour on Linux older than 2.6.27 XXX with SOCK_CLOEXEC? XXX

New functions:

If accept4() is called on Linux older than 2.6.28, accept4() returns -1 (fail) and errno is set to ENOSYS.

Links

Links:

Python issues:

Ruby:

Footnotes

.. [#subprocess_close] On UNIX since Python 3.2, subprocess.Popen() closes all file descriptors by default: close_fds=True. It closes file descriptors in range 3 inclusive to local_max_fd exclusive, where local_max_fd is fcntl(0, F_MAXFD) on NetBSD, or sysconf(_SC_OPEN_MAX) otherwise. If the error pipe has a descriptor smaller than 3, ValueError is raised.

More information about the Python-Dev mailing list