[Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets (original) (raw)

Victor Stinner victor.stinner at gmail.com
Sat Jul 6 14:04:13 CEST 2013

2013/7/6 Charles-François Natali <cf.natali at gmail.com>:

I've read your "Rejected Alternatives" more closely and Ulrich Drepper's article, though I think the article also supports adding a blocking (default True) parameter to open() and os.open(). If you try to change that default on a platform where it doesn't work, an exception should be raised. Contrarily to close-on-exec, non-blocking only applies to a limited type of files (e.g. it doesn't work for regular files, which represent 90% of open() use cases).

What do you mean by "does not work"? On Linux, O_NONBLOCK flag can be set on regular files, sockets, pipes, etc.

Example with a regular file on Linux 3.9:

smithers$ python3 Python 3.3.0 (default, Sep 29 2012, 22:07:38)

from fcntl import * import os f=open("/etc/issue", "rb") fd=f.fileno() flags=fcntl(fd, FGETFL) fcntl(fd, FSETFL, flags|os.ONONBLOCK) 0 fcntl(fd, FGETFL) & os.ONONBLOCK 2048 f.read(10) b'Fedora rel'

For example, asyncore.file_dispatcher() uses fcntl to set the O_NONBLOCK flag. (The asyncore module is probably the only module of the stdlib which would benefit of a new os.set_blocking() function.)

Also, one of the main reasons for exposing close-on-exec in open()/socket() etc is to make it possible to create file descriptors with the close-on-exec flag atomically, to prevent unwanted FD inheritance especially in multi-threaded code. And that's not necessary for the non-blocking parameter.

In the review of PEP 433 on this mailing list, Martin von Loewis said that the PEP does guarantee the atomicity. The implementation of the PEP falls back on ioctl or fcntl to set the flag on old Linux versions (or if the OS does not support setting O_CLOEXEC flag when creating a file descriptor). The GIL is released during system calls, so another thread can call execv().

That's why I mentionned explicitly in the PEP 446 that it does not offer any atomicity guarantee.

But open(filename, blocking=False) on Linux has at least one advantage over f=open(filename); os.set_blocking(f.fileno()): it only uses one syscall (versus 2 or 3 syscalls).

Victor

More information about the Python-Dev mailing list