[Python-Dev] Require loaders set package and loader (original) (raw)

Glyph glyph at twistedmatrix.com
Sun Apr 15 23:12:28 CEST 2012

On Apr 14, 2012, at 3:32 PM, Guido van Rossum wrote:

Funny, I was just thinking about having a simple standard API that will let you open files (and list directories) relative to a given module or package regardless of how the thing is loaded.

Twisted has such a thing, mostly written by me, called twisted.python.modules.

Sorry if I'm repeating myself here, I know I've brought it up on this list before, but it seems germane to this thread. I'd be interested in getting feedback from the import-wizards participating in this thread in case it is doing anything bad (in particular I'd like to make sure it will keep working in future versions of Python), but I think it may provide quite a good template for a standard API.

The code's here: <http://twistedmatrix.com/trac/browser/trunk/twisted/python/modules.py>

The API is fairly simple.

from twisted.python.modules import getModule e = getModule("email") # get an abstract "module" object (un-loaded) e PythonModule<'email'> walker = e.walkModules() # walk the module hierarchy walker.next() PythonModule<'email'> walker.next() PythonModule<'email._parseaddr'> walker.next() # et cetera PythonModule<'email.base64mime'> charset = e["charset"] # get the 'charset' child module of the 'e' package charset.filePath FilePath('.../lib/python2.7/email/charset.py') charset.filePath.parent().children() # list the directory containing charset.py

Worth pointing out is that although in this example it's a FilePath, it could also be a ZipPath if you imported stuff from a zipfile. We have an adapter that inspects path_importer_cache and produces appropriately-shaped filesystem-like objects depending on where your module was imported from. Thank you to authors of PEP 302; that was my religion while writing this code.

You can also, of course, ask to load something once you've identified it with the traversal API:

charset.load() <module 'email.charset' from '.../lib/python2.7/email/charset.pyc'>

You can also ask questions like this, which are very useful when debugging setup problems:

ifaces = getModule("twisted.internet.interfaces") ifaces.pathEntry PathEntry<FilePath('/Domicile/glyph/Projects/Twisted/trunk')> list(ifaces.pathEntry.iterModules()) [PythonModule<'setup'>, PythonModule<'twisted'>]

This asks what sys.path entry is responsible twisted.internet.interfaces, and then what other modules could be loaded from there. Just 'setup' and 'twisted' indicates that this is a development install (not surprising for one of my computers), since site-packages would be much more crowded.

The idiom for saying "there's a file installed near this module, and I'd like to grab it as a string", is pretty straightforward:

from twisted.python.modules import getModule mod = getModule(name).filePath.sibling("my-file").open().read()

And hopefully it's obvious from this idiom how one might get the pathname, or a stream rather than the bytes.

-glyph -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20120415/0112594d/attachment.html>

More information about the Python-Dev mailing list