[Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info (original) (raw)

Stefan Drees stefan at drees.name
Mon May 13 14:47:21 CEST 2013

Hi Ben,

Am 13.05.13 14:25, schrieb Ben Hoyt:

...It's not yet production-ready, and is basically still in API and performance testing stage. ...

In any case, I really like the API (thanks mostly to Nick Coghlan), and performance is great, even with DirEntry being written in Python. PERFORMANCE: On Windows I'm seeing that scandir.walk() on a large test tree (see benchmark.py) is 8-9 times faster than os.walk(), and on Linux it's 3-4 times faster. Yes, it is that much faster, and yes, those numbers are real. :-) Please critique away. At this stage it'd be most helpful to critique any API or performance-related issues ...

you asked for critique, but the performance seems to be also 2-3 times speedup (as stated by benchmark.py) on mac osx 10.8.3 (on MacBook Pro 13 inch, start of 2011, solid state disk) with python 2.7.4 (the homebrew one):

$> git clone git://github.com/benhoyt/scandir.git $> cd scandir && python setup.py install $> python benchmark.py USING FAST C version Creating tree at benchtree: depth=4, num_dirs=5, num_files=50 Priming the system's cache... Benchmarking walks on benchtree, repeat 1/3... Benchmarking walks on benchtree, repeat 2/3... Benchmarking walks on benchtree, repeat 3/3... os.walk took 0.104s, scandir.walk took 0.031s -- 3.3x as fast

$> python benchmark.py -s USING FAST C version Priming the system's cache... Benchmarking walks on benchtree, repeat 1/3... Benchmarking walks on benchtree, repeat 2/3... Benchmarking walks on benchtree, repeat 3/3... os.walk size 226395000, scandir.walk size 226395000 -- equal os.walk took 0.246s, scandir.walk took 0.125s -- 2.0x as fast

So for now, all well and thank you.

All the best,

Stefan.

More information about the Python-Dev mailing list