[Python-Dev] C extension import time (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Fri Oct 11 14:51:56 CEST 2013

Le Fri, 11 Oct 2013 14:24:29 +0200, Stefan Krah <stefan at bytereef.org> a écrit :

Hi,

recently there has been some talk about reducing import times. It seems that the current import strategy for C extensions (i.e. importing the extension at the bottom of the .py file) is quite slow:

==================== import sys for i in range(10000): import decimal del sys.modules('decimal') del sys.modules('decimal') ====================

Try the following:

$ ./python -m timeit -s "modname='decimal'; import sys"
"import(modname); del sys.modules[modname]" 1000 loops, best of 3: 2.21 msec per loop

$ ./python -m timeit -s "modname='_decimal'; import sys"
"import(modname); del sys.modules[modname]" 10000 loops, best of 3: 112 usec per loop

Now compare with the time taken to simply find the module loader:

$ ./python -m timeit -s "modname='_decimal'; import importlib"
"importlib.find_loader(modname)" 10000 loops, best of 3: 87.2 usec per loop

So find_loader() is the bigger contributor here. Note that find_loader() can have varying performance:

$ touch empty.py $ ./python -m timeit -s "modname='empty'; import importlib"
"importlib.find_loader(modname)" 10000 loops, best of 3: 24.4 usec per loop

When trying to improve import performance, I noticed that merely diagnosing the bottlenecks is not always easy.

Regards

Antoine.

More information about the Python-Dev mailing list