[Python-Dev] Importing .pyc in -O mode and vice versa (original) (raw)

"Martin v. Löwis" martin at v.loewis.de
Mon Nov 6 07:49:14 CET 2006

Greg Ewing schrieb:

That should never be better: the system will cache the directory blocks, also, and it will do a better job than Python will. If that's really the case, then why do discussions of how improve Python startup speeds seem to focus on the number of stat calls made?

A stat call will not only look at the directory entry, but also look at the inode. This will require another disk access, as the inode is at a different location of the disk.

Also, cacheing isn't the only thing to consider. Last time I looked at the implementation of unix file systems, they mostly seemed to do directory lookups by linear search. Unless that's changed a lot, I have a hard time seeing how that's going to beat Python's highly-tuned dictionaries.

It depends on the file system you are using. An NTFS directory lookup is a B-Tree search; NT has not been doing linear search since its introduction 15 years ago. Linux only recently started doing tree-based directories with the introduction of ext4. However, Linux' in-memory directory cache (the dcache) doesn't need to scan over the directory block structure; not sure whether it uses linear search still.

For a small directory, the difference is likely negligible. For a large directory, the cost of reading in the entire directory might be higher than the savings gained from not having to search it. Also, if we do our own directory caching, the question is when to invalidate the cache.

Regards, Martin

More information about the Python-Dev mailing list