[Python-Dev] Better support for consuming vendored packages (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sat Mar 24 05:29:24 EDT 2018

On 23 March 2018 at 02:58, Gregory Szorc <gregory.szorc at gmail.com> wrote:

I'd like to start a discussion around practices for vendoring package dependencies. I'm not sure python-dev is the appropriate venue for this discussion. If not, please point me to one and I'll gladly take it there.

Since you mainly seem interested in the import side of things (rather than the initial vendoring process), python-ideas is probably the most suitable location (we're not at the stage of a concrete design proposal that would be appropriate for python-dev, and this doesn't get far enough into import system arcana to really need to be an import-sig discussion rather than a python-ideas one).

What we've done is effectively rename the "shrubbery" package to "knights.vendored.shrubbery." If a module inside that package attempts an import shrubbery.x, this could fail because "shrubbery" is no longer the package name. Or worse, it could pick up a separate copy of "shrubbery" somewhere else in sys.path and you could have a Frankenstein package pulling its code from multiple installs. So for this to work, all package-local imports must be using relative imports. e.g. from . import_ _x.

If it's the main application doing the vendoring, then the following kind of snippet can be helpful:

from knights.vendored import shrubbery
import sys
sys.path["shrubbery"] = shrubbery

So doing that kind of aliasing on a process-wide basis is already possible, as long as you have a point where you can inject the alias (and by combining it with a lazy importer, you can defer the actual import until someone actually uses the module).

Limiting aliasing to a particular set of modules doing imports would be much harder though, since we don't pass that information along (although context variables would potentially give us a way to make it available without having to redefine all the protocol APIs)

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20180324/c8e08ce5/attachment.html>

More information about the Python-Dev mailing list