[Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Tue May 14 10:37:35 CEST 2013

Le Tue, 14 May 2013 10:41:01 +1200, Ben Hoyt <benhoyt at gmail.com> a écrit :

> I'd to see the numbers for NFS or CIFS - stat() can be brutally slow > over a network connection (that's why we added a caching mechanism > to importlib).

How do I know what file system Windows networking is using? In any case, here's some numbers on Windows -- it's looking pretty good! This is with default DEPTH/NUMDIRS/NUMFILES on a LAN: Benchmarking walks on \anothermachine\docs\Ben\bigtree, repeat 3/3... os.walk took 11.345s, scandir.walk took 0.340s -- 33.3x as fast And this is on a VPN on a remote network with the benchmark.py values cranked down to DEPTH = 3, NUMDIRS = 3, NUMFILES = 20 (because otherwise it was taking far too long): Benchmarking walks on \ben1.titanmt.local\c$\dev\scandir\benchtree, repeat 3/3... os.walk took 122.310s, scandir.walk took 5.452s -- 22.4x as fast If anyone can run benchmark.py on Linux / NFS or similar, that'd be great. You'll probably have to lower DEPTH/NUMDIRS/NUMFILES first and then move the "benchtree" to the network file system to run it against that.

Why does your benchmark create such large files? It doesn't make sense.

Regards

Antoine.

More information about the Python-Dev mailing list