[Python-Dev] Should there be a way or API for retrieving from a code object a loader method and package file where the code comes from? (original) (raw)

Paul Moore p.f.moore at gmail.com
Tue Dec 23 17:00:25 CET 2008

2008/12/23 Nick Coghlan <ncoghlan at gmail.com>:

Finding a loader given only a pseudo-filename and no module is actually possible in the specific case of zipimport, but is still pretty obscure at this point in time:

1. Scan sys.path looking for an entry that matches the start of the pseudo-filename (remembering to use os.path.normpath). 2. Once such a path entry has been found, use PEP 302 to find the associated importer object (the undocumented pkgutil.getimporter function does exactly that - although, as with any undocumented feature, the promises of API compatibility across major version changes aren't as strong as they would be for an officially documented and supported interface). 3. Hope that the importer is one like zipimport that allows getdata() to be invoked directly on the importer object, rather than only providing it on a separate loader object after the module has been loaded. If it needs a real loader instead of just the importer, then you're back to the original problem of needing a module or package name (or globals dictionary) in addition to the pseudo filename.

There were lots of proposals tossed around on python-dev at the time PEP 302 was being developed, which might have made all this easier. Most, if not all, were killed by backward compatibility requirements.

I have some hopes that when Brett completes his "import in Python" work, that will add sufficient flexibility to allow people to experiment with all of this machinery, and ultimately maybe move forward with a more modular import mechanism. But the timescales for Brett's changes won't be until at least Python 3.1, and it'll be a release or two after that before any significant change can be eased in in a compatible manner. That's going to take a lot of energy on someone's part.

Paul.

PS One of these days, I'm going to write an insanely useful importer which takes the least-convenient option wherever PEP 302 allows flexibility. It'll be adopted by everyone because it's so great, and all the software that currently makes unwarranted assumptions about importers will break and get fixed to support it because otherwise its users will rebel, and we'll live in a paradise where everything follows the specs to the letter. Oh, yes, and I'm going to win the lottery every week for the next month :-)

PPS Seriously, setuptools and the adoptions of eggs has pushed a lot of code to be much more careful about unwarranted assumptions that code lives in the filesystem. That's an incredibly good thing, and very hard to do right (witness the setuptools "zip_safe" parameter which acts as a get-out clause). Much kudos to setuptools for getting as far as it has.

More information about the Python-Dev mailing list