[Python-Dev] Startup time (original) (raw)

Jeremy Hylton jeremy@zope.com
14 May 2003 11:55:57 -0400

I don't know if this thread came to any conclusion. I did the same strace that everyone else has reported on, and I've included a summary of the results here. I have one directory in my PYTHONPATH, which affects the number of directories that are searched for each imported module.

Comparing Python 2.3 current CVS with Python 2.2 CVS, I see the following system call counts (limited to top 6 in 2.3).

     2.3   2.2

open 305 104 stat64 102 44 fstat64 74 34 read 71 30 rt_sig... 69 68 brk 62 74

When a single module is imported from the standard library, Python 2.2 looks in 10 different places. Specifically, it looks for five different files in two different directories -- PYTHONPATH and the std library directory. For files that aren't found (e.g. sitecustomize), it looks in 25 places (5 files x 5 directories). Interesting to note that PYTHONPATH directory is not searched for sitecustomize.

In Python 2.3, the standard library module requires 15 lookups because /usr/local/lib/python23.zip is added to the path before the std library directory. The failed lookup of sitecustomize takes 35 lookups, because PYTHONPATH and python23.zip are now on the path.

The list of attempted imports is much larger in 2.3 than in 2.2.

-- 2.3 -- -- 2.2 -- future codecs copy_reg copy_reg encodings encodings/init encodings/aliases encodings/iso_8859_15 exceptions linecache os os posixpath posixpath re site site sitecustomize sitecustomize sre sre_compile sre_constants sre_parse stat stat string strop types types UserDict UserDict warnings 22 total 7 total

The increase in open, stat64, and fstat64 all seem consistent with a 3x increase in the number of modules searched for.

The use of re in the warnings module seems the primary culprit, since it pulls in re, sre and friends, string, and strop.

Jeremy