[Python-Dev] PEP 471 (scandir): Poll to choose the implementation (full C or C+Python) (original) (raw)

Victor Stinner victor.stinner at gmail.com
Fri Feb 13 12:34:01 CET 2015

2015-02-13 12:27 GMT+01:00 Serhiy Storchaka <storchaka at gmail.com>:

On 13.02.15 12:07, Victor Stinner wrote:

* C implementation: scandir is at least 3.5x faster than listdir, up to 44.6x faster on Windows Results on Windows was obtained in the becnhmark that doesn't drop disk caches and runs listdir before scandir.

The benchmark code is here: http://bugs.python.org/file38120/bench_scandir2.py

Eeach test is repeated 5 times, I compared the duration of the fastest call. A test calls the function 5 times in a loop.

Anyway, it's not the first call to listdir() which fills the disk cache, but the call to count_entries() (which is implemented with os.scandir).

So is there any issue in the benchmark script or not?

On Linux, you can use "bench_nocache" (and "bench_nostat_nocache") which flushs the cache: it writes "3" into /proc/sys/vm/drop_caches. scandir is always faster when the disk cache is flushed.

Victor

More information about the Python-Dev mailing list