[Python-Dev] PEP 364, Transitioning to the Py3K standard library (original) (raw)

Barry Warsaw barry at python.org
Thu Mar 8 00:43:15 CET 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On Mar 7, 2007, at 4:36 PM, Brett Cannon wrote:

>> Third party package renaming is also supported, via several public >> interfaces accessible by any Python module. >> > > I guess a .pth file could install the mappings for the third-party > modules.

How would that work? .pth files don't execute arbitrary Python code, It does through imports, though.

I think it's important to import on demand only though. I should
probably make that clear in the PEP . IOW, "import email"
should not by side-effect import all sub-modules just because there's
a remapping involved. This actually would be quite easy to do, but
it's one of the reasons why the Python 2.5 email package hack goes
through so much trouble.

but it seems like that would be necessary to make this work. OTOH, your suggestion does make me think that it might be possible to piggyback the module name mappings onto .pth file processing. It already understand the 'import' command, so why not a 'rename' command? Not needed. Just put your mapping code in a module in your package and have a .pth file import that module.

Think about what that big dictionary would look like. It would essentially be like a text file with mappings one per line, but you'd have a lot of extra goo on each line (quotes, colons, and commas). I thought it was important to decentralize the mappings and then to put them in the simplest format that would serve the purpose. Right, but I am just not a big special-purpose format guy. =) Plus mappings for third-parties can be kept to a single file for the module they import by keeping the dict in that file.

If you went with the dictionary approach, then you wouldn't need
the .pth file. I guess a question for either approach is what you do
if the top-level module is renamed. E.g. if we rename Carbon to
carbon, where do we put the remapping for this? You can't keep both
directories because of case-insensitive file systems (this has
already been a "fun" pep to develop on a Mac ;). If we narrow the
scope to only the official stdlib, then it's probably not an issue
because we can stick it anywhere and arrange for it to get loaded.

I guess I would want to minimize the API in case we decide to directly integrate this into import itself so we don't have to have both a metapath and pathimportercache object. I think the solution works for PEP 3108 and the mail package but I think I would a more integrated solution if we went all out in terms of providing the support outside of the stdlib (at least eventually).

It depends on how explicit or magical you want the hooks to be. More
magic is more convenient to hook into, but perhaps harder to decipher.

If people really don't like the separate data file, and really want to just use a Python dictionary, then the way to keep the decentralization would be to reserve a * symbol in the package namespace that contained this remapping. Something like renames = {...}. I think the .pth file solution alleviates this need.

At the cost of immediately importing all sub-modules when the package
is imported. Do you agree that lazy loading is a requirement we
should keep?

> If you are going to have the object accessible from outside > sys.pathimportercache I would also add an attribute that contains > the magic string used on sys.path.

Of course, that magic string is also available via oldlib.magic. I tend to think it won't be necessary, but if people really want it, I can see putting the magic string as an attribute on the OldStdlibLoader object, e.g. sys.stdlibremapper.importercachekey. Yeah, that was what I was thinking of. When you manipulate sys.path you will need to be aware of it and what it is named.

Good point. Yes, I like putting the magic key on the remapper
object. That does give us a bit of a chicken and egg though because
until you know the magic string, you can't know the
path_importer_cache key, so I think putting the remapper in sys is
useful.

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRe9OFHEjvBPtnXfVAQIVhwP+Oloyxi/ldlbQ7RM2ifmgN03jc/SYN5Ht suxsXiK3xwiCU5B/J6uAKDSol/WtxKV/EI2yN42xJIUFV0B0C8DCLHDZHLDAI67V dfsJMAV+wgn8RjPVSuEw9vOFaVaoyIVvdDn+XMKKfKX6YOBnjLpnLaZJtEVx2QMg ZigImiWp3VI= =EHOG -----END PGP SIGNATURE-----

More information about the Python-Dev mailing list