[Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info (original) (raw)

Ronald Oussoren ronaldoussoren at mac.com
Fri May 10 16:20:29 CEST 2013

On 10 May, 2013, at 15:54, Antoine Pitrou <solipsis at pitrou.net> wrote:

Le Fri, 10 May 2013 15:46:21 +0200, Christian Heimes <christian at python.org> a écrit :

Am 10.05.2013 14:16, schrieb Antoine Pitrou: But what if some systems return more than the file type and less than a full stat result? The general problem is POSIX's terrible inertia. I feel that a stat result with some None fields would be an acceptable compromise here.

POSIX only defines the dino and dname members of struct dirent. Linux, BSD and probably some other platforms also happen to provide dtype. The other members of struct dirent (dreclen, dnamlen) aren't useful in Python space by themselves. dtype and stmode aren't compatible in any way. As you know stmode also contains POSIX permission information. The file type is encoded with a different set of bits, too. Future file types aren't mapped to SIF* constants for stmode. Thank you and Ronald for clarifying. This does make the API design a bit bothersome. We want to expose as much information as possible in a cross-platform way and with a flexible granularity, but doing so might require a gazillion of namedtuple fields (platonically, as much as one field per stat bit).

One field per stat bit is overkill, file permissions are well known enough to keep them as a single item.

Most if not all uses of the st_mode field can be covered by adding just "filetype" and "permissions" fields. That would also make it possible to use stat_result in os.scandir() without loosing information (it would have filetype != None and permissions and st_mode == None on systems with d_type).

For dino you also need the device number from the directory because the inode is only unique within a device. But hopefully you've already stat'ed the directory ;)

Why? There's no need to stat the directory when implementing os.walk using os.scandir (for systems that return filetype information in the API used by os.scandir). Anyway, setting st_ino in the result of os.scandir is harmless, even though using st_ino is uncommon.

Getting st_dev from the directory isn't good anyway, for example when using rebind mounts to mount a single file into a different directory (which is a convenient way to make a configuration file available in a chroot environment)

Ronald

Regards Antoine.

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

More information about the Python-Dev mailing list