[Python-Dev] New and Improved Import Hooks (original) (raw)

Guido van Rossum guido@python.org
Thu, 05 Dec 2002 08:24:04 -0500

> Why don't you care about the backwards incompatibilities?

There is no strict backwards incompatibility: Programs that work now continue to work, as they will find only strings on sys.path. You only get problems if you actually add a zipfile to sys.path.

I've written code that searches the path for data files. This has been brought up as an example before; Zope 3 for example looks for page templates and configuration files on the path. I'd like such code to continue to work even if the standard library is loaded from a zip file. Such code is typically robust if one of the sys.path entries doesn't refer to a valid directory; but it may only catch os.error, and not the TypeError resulting from attempts at string concatenation.

Obviously if the data files get stored in the zip file too, the code searching for it needs to be made zip file aware; but if zip files are used to provide other libraries, that shouldn't break code that looks for data files along sys.path.

It is unclear to me what the typical breakage would be, and whether it could be solved in most cases by having the import hooks a string subclass.

A string subclass would work. But it sounds like the requirement to subclass str makes writing hooks more difficult compared to a mechanism that places the hook in a separate hook registry.

I can imagine a number of potential problems. Code may choke if it encounters a Unicode string on sys.path (which is possible already), it may choke if it encounters a non-ASCII byte string on sys.path, it may choke if it encounters a string that is not a directory name (in which case you simply can't add zipfile imports without breaking backwards compatibility).

The main breakage I am intersted in avoiding is brekage when you concatenate an ASCII string with a sys.path item.

--Guido van Rossum (home page: http://www.python.org/~guido/)