Issue 26781: os.walk max_depth - Python tracker (original) (raw)

Created on 2016-04-16 14:11 by palaviv, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
os-walk-max-depth.patch palaviv,2016-04-16 14:11 review
os-walk-max-depth-2.patch palaviv,2016-04-16 18:32 After SilentGhost review review
os.py andrekorol,2017-07-17 13:25 os.py with lwalk function implemented

| Repositories containing patches | | | | | -------------------------------------------------------------------------------------------------------- | | | | | https://github.com/andrekorol/cpython/ | | | |

Messages (6)
msg263556 - (view) Author: Aviv Palivoda (palaviv) * Date: 2016-04-16 14:11
I am suggesting to add max_depth argument to os.walk. I think this is very useful for two cases. The trivial one is when someone wants to walk on a directory tree up to specific depth. The second one is when you follow symlinks and wish to avoid infinite loop. The patch add the max_depth both to os.walk and os.fwalk.
msg263565 - (view) Author: (random832) Date: 2016-04-16 15:47
Wouldn't the "symlink infinite loop" case be better handled by making it track where it's already been? This can be done by inode and dev number on Unix; I'm not sure what equivalent exists on Windows (though symlinks are uncommon on Windows) but you could fall back on realpath.
msg263579 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-04-16 19:15
os.walk() allows more flexible control. for root, dirs, files in os.walk(top): if is_too_deep(root): dirs.clear() continue ... You can even walk up to different depth on different parts of the tree. You can limit walking not by the directory depth, but by the length of the path, or the number of links in the path, or what-you-need. Adding separate parameters for all this particular cases is not practical. I think there is a little need in this feature.
msg263861 - (view) Author: Aviv Palivoda (palaviv) * Date: 2016-04-20 20:35
I am not sure about the wide use of this feature but I do think this is a nice feature to add to os.walk. I can see how you can implement is_too_deep by counting the number of separators in the path. However I don't think it is trivial to do that. In addition doing the same when topdown is False will not work. As for the other limitations you suggested I think they are less trivial then limiting the recursion depth. As for the "symlink infinite loop" I agree that it will be better to do that by tracking the paths we have already visited but avoiding infinite loop is just a bonus that you get from this feature.
msg298510 - (view) Author: André Rossi Korol (andrekorol) * Date: 2017-07-17 13:25
I proposed a new function called lwalk(level walk) that recurses only to a certain level of depth: http://bugs.python.org/issue30942 It is implemented in os.py and calls os.walk, but making sure it recurses only to a selected level of depth. If it is accepted I could send a Pull Request with the lwalk function implemented in os.py.
msg298524 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-07-17 14:15
> I think there is a little need in this feature. I concur with Serhiy and think we're better-off without this proposal. Marking this as closed.
History
Date User Action Args
2022-04-11 14:58:29 admin set github: 70968
2017-07-17 14:15:30 rhettinger set status: open -> closednosy: + rhettingermessages: + resolution: rejectedstage: resolved
2017-07-17 13:25:48 andrekorol set files: + os.pynosy: + andrekorolmessages: + hgrepos: + hgrepo371
2017-07-17 12:40:03 r.david.murray link issue30942 superseder
2016-04-20 20:35:03 palaviv set messages: +
2016-04-16 19:15:46 serhiy.storchaka set nosy: + serhiy.storchakamessages: +
2016-04-16 18:32:48 palaviv set files: + os-walk-max-depth-2.patch
2016-04-16 15:47:51 random832 set nosy: + random832messages: +
2016-04-16 14:11:10 palaviv create