[Python-Dev] Better support for consuming vendored packages (original) (raw)

Gregory Szorc gregory.szorc at gmail.com
Thu Mar 22 15:00:59 EDT 2018

On 3/22/2018 10:48 AM, Oleg Broytman wrote:

Hi!

On Thu, Mar 22, 2018 at 09:58:07AM -0700, Gregory Szorc <gregory.szorc at gmail.com> wrote: Not all consumers of Python packages wish to consume Python packages in the common pip install <package> IMO pip is for developers. To package and distribute end-user applications there are rpm, dpkg/deb, PyInstaller, cxFreeze, py2exe (+ installer like NSIS or InnoSetup), py2app, etc... Most of them pack a copy of Python interpreter and necessary parts of stdlib, so there is no problem with sys.path and wrong imports.

Yes, there are tools to create standalone packages. Some even bundle a Python install so the execution environment is more deterministic. These are great ways to distribute Python applications!

However, if you are a Python application that is maintained by your distribution's package manager, you pretty much must use a Python installed by the system package manager. And that leaves us with the original problem of an undefined execution environment. So packaging tools for standalone Python applications only work if you control the package distribution channel. If you are a successful Python application that is packaged by your distro, you lose the ability to control your own destiny and must confront these problems for users who installed your application through their distro's package manager. i.e. the cost of success for your Python application is a lot of pain inflicted by policies of downstream packagers.

Also, not vendoring dependencies puts the onus on downstream packagers to deal with those dependencies. There can be package version conflicts between various packaged Python applications ("dependency hell"). Vendoring dependencies under application-local package names removes the potential for version conflicts.

It's worth noting that some downstream packagers do insist on unbundling dependencies. So they may get stuck with work regardless. But if you vendor dependencies, a downstream packager is at least capable of packaging a Python application without having to deal with "dependency hell."

Maybe what I'm asking for here is import machinery where an application can forcefully limit or influence import mechanisms for modules in a certain package. But this seems difficult to achieve given the constraint of a single, global modules namespace (sys.modules) per interpreter.

More information about the Python-Dev mailing list