Brett Cannon wrote:
> Actually it's four: name/__init__.py, name/__init__.pyc, name.py, and
> then name.pyc. And just so people have terminology to go with all of
> this, this search is what the finder does to say whether it can or
> cannot handle the requested module.

Huh, I thought we checked for the directory first and only then checked
for the __init__ module within it (hence the generation of ImportWarning
when we don't find __init__ after finding a correctly named directory).
So a normal miss (i.e. no directory) only needs one stat call.

(However, I'll grant that I haven't looked at this particular chunk of
code in a fairly long time, so I could easily be wrong).

Robert raises a good point about the checks for extension modules as
well - we should get an accurate count here so Barry's PEP can pitch the
proportional reduction in stat calls accurately.

Here are the details (from Python/import.c:find_module) assuming that everything has failed to the point of trying for the implicit sys.path importers:
">

(original) (raw)



On Sun, Feb 28, 2010 at 12:46, Nick Coghlan <ncoghlan@gmail.com> wrote:

Brett Cannon wrote:
> Actually it's four: name/\_\_init\_\_.py, name/\_\_init\_\_.pyc, name.py, and
> then name.pyc. And just so people have terminology to go with all of
> this, this search is what the finder does to say whether it can or
> cannot handle the requested module.

Huh, I thought we checked for the directory first and only then checked
for the \_\_init\_\_ module within it (hence the generation of ImportWarning
when we don't find \_\_init\_\_ after finding a correctly named directory).
So a normal miss (i.e. no directory) only needs one stat call.

(However, I'll grant that I haven't looked at this particular chunk of
code in a fairly long time, so I could easily be wrong).

Robert raises a good point about the checks for extension modules as
well - we should get an accurate count here so Barry's PEP can pitch the
proportional reduction in stat calls accurately.

Here are the details (from Python/import.c:find\_module) assuming that everything has failed to the point of trying for the implicit sys.path importers:

stat\_info = stat(name)
if stat\_info.exists and stat\_info.is\_dir:
if stat(name/\_\_init\_\_.py) || stat(name/\_\_init\_\_.pyc):
load(name)
else:
for ext in ('.so', 'module.so', '.py', 'pyc'): # Windows has an extra check for .pyw files.
if open(name + ext):
load(name)

So there are a total of five to six depending on the OS (actually, VMS goes up to eight!) before a search path is considered not to contain a module.

And thanks to doing this I realized importlib is not stat'ing the directory first which should fail faster than checking for the \_\_init\_\_ files every time.

-Brett





Cheers,
Nick.

\--
Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
\---------------------------------------------------------------