[Python-Dev] Complexity of import.c (original) (raw)

James C. Ahlstrom jim@interet.com
Mon, 09 Dec 2002 12:24:11 -0500

I see that my patch to import.c has been criticized as adding to the complexity of an already complex module. I don't entirely agree. If you want to have the standard library available as a zip module, you must have zip file import available in C. The patch I proposed in PEP 273 was a minimalist patch; it adds as little as possible while still keeping package import and preserving all import semantics.

IMHO, the real complexity of import.c comes not from the double loop for path in sys.path: for suffix in (".pyc", ".pyo",...): fopen(...) and not from the caching of zip archive names. It comes from the use of import.c to perform package imports. Package imports are performed by creating recursive calls to functions that would otherwise be flat, and by replacing sys.path with the package path. This is darned confusing.

At the time PEP 273 was discussed, I proposed moving package imports out of import.c into a Python module such as Gordon's iu.py or Greg's imputil.py. This was rejected because package users feared that package imports would be slowed down. The speed of Python startup and import was a concern.

I understand that people want to generalize zip imports, and create a better import hook mechanism. Some feel that now is the time to do it.

This is my humble opinion:

Replacing import.c plus zip import with an equally complex import.c is a fundamental mistake. The current import.c is understood (by few), and the zip file import patch adds the minimum required. If a different import.c is accepted, then it should be MUCH simpler, and support only flat directory searches and zip directories. Package imports and import hooks should be moved to a Python module such as Gordon's iu.py. And import.c must bootstrap iu.py and all its imports from either a directory search or a zip file.

An improved import hook mechanism should have its own PEP.

I tried my best not to break the Macintosh port, which has a lot of special code. A replacement import.c should do the same.

My patch modified many other Python .c and .py files to solve several difficult bootstrap problems. If you replace just import.c you will get a painful lesson in bootstrapping. You probably need these other patches even if you reject my import.c patch.

Jim Ahlstrom