Issue 13739: os.fdlistdir() is not idempotent (original) (raw)
Created on 2012-01-08 16:26 by neologix, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (7)
Author: Charles-François Natali (neologix) * 
Date: 2012-01-08 16:26
After a call to fdlistdir(), another call to fdlistdir() on the same file handle (but using a different FD, since the FD passed to fdlistdir() is closed) will return an empty list:
""" $ cat ~/test_fdlistdir.py import os import sys
fd = os.open(sys.argv[1], os.O_RDONLY)
print(os.fdlistdir(os.dup(fd))) print(os.fdlistdir(os.dup(fd)))
os.close(fd) $ ./python ~/test_fdlistdir.py /tmp/ ['pulse-B1FebW397VI5', 'ksocket-kdm', 'etc', 'kde-cf', 'ksocket-cf', 'test_posix.py', '.X0-lock', 'kde-kdm', 'akonadi-cf.k6y52j', 'ssh-iSFleEAS1243', '.ICE-unix', '.X11-unix'] [] """
That's because fdopendir()/readdir doesn't reset the FD offset. It's documented by POSIX: """ The file offset associated with the file descriptor at the time of the call determines which entries are returned. """
That's rather suprising (I got bitten while trying to write a test for #13734). I see two options:
- rewind the directory stream in fdlistdir()
- document this
Here's a patch for option 1.
Author: Antoine Pitrou (pitrou) *
Date: 2012-01-08 17:18
I see two options:
- rewind the directory stream in fdlistdir()
- document this
Here's a patch for option 1.
Agreed with that, and ok with the patch :)
Author: Ross Lagerwall (rosslagerwall)
Date: 2012-01-08 17:32
I see two options:
- rewind the directory stream in fdlistdir()
- document this
Here's a patch for option 1.
Yeah, looks good.
Author: Roundup Robot (python-dev) 
Date: 2012-01-08 17:34
New changeset 7b2a178c028b by Charles-François Natali in branch 'default': Issue #13739: In os.listdir(), rewind the directory stream (so that listdir() http://hg.python.org/cpython/rev/7b2a178c028b
Author: Roundup Robot (python-dev)
Date: 2012-01-08 18:07
New changeset 36f2e236c601 by Charles-François Natali in branch 'default': Issue #13739: It's simpler and more direct to call rewinddir() at the http://hg.python.org/cpython/rev/36f2e236c601
Author: Antoine Pitrou (pitrou) *
Date: 2012-01-08 19:13
For some reason, the second changeset broke the OpenIndiana buildbots: http://www.python.org/dev/buildbot/all/builders/AMD64%20OpenIndiana%203.x/builds/2485
Also, wouldn't it be better to call rewinddir with the GIL released? (although I agree rewinddir shouldn't be expensive)
Author: Charles-François Natali (neologix) *
Date: 2012-01-08 19:29
For some reason, the second changeset broke the OpenIndiana buildbots:
I have absolutely no idea of why this doesn't work. I suspect rewinddir() is a noop on OpenIndiana if readdir() hasn't been called. I'll revert this commit.
Also, wouldn't it be better to call rewinddir with the GIL released? (although I agree rewinddir shouldn't be expensive)
rewinddir() just changes a pointer, and calls - or ought to call - lseek() on the FD. This should be fast, since no I/O is involved (lseek() is not documented to return EINTR, for example). Releasing the GIL has a cost :-)
History
Date
User
Action
Args
2022-04-11 14:57:25
admin
set
github: 57948
2012-01-09 19:42:29
neologix
set
status: open -> closed
resolution: fixed
stage: resolved
2012-01-08 19:29:03
neologix
set
messages: +
2012-01-08 19:13:47
pitrou
set
messages: +
2012-01-08 18:07:42
python-dev
set
messages: +
2012-01-08 17:34:43
python-dev
set
nosy: + python-dev
messages: +
2012-01-08 17:32:07
rosslagerwall
set
messages: +
2012-01-08 17🔞31
pitrou
set
messages: +
2012-01-08 16:26:25
neologix
create