[Python-Dev] Status of packaging in 3.3 (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Thu Jun 21 15:29:23 CEST 2012

On Thu, Jun 21, 2012 at 10:51 PM, Chris McDonough <chrism at plope.com> wrote:

Is it reasonable to even assume there is only one-sys.path-to-rule-them-all? And that users install "the set of libraries they need" into a common place?  This quickly turns into failure, because Python is used for many, many tasks, and those tasks sometimes *require conflicting versions of libraries*.  This is the root cause of why virtualenv exists and is popular.

And why I'm very happy to see pyvenv make it's way into the standard library :)

I care about deploying Python-based applications to many platforms.  You care about deploying multilanguage-based applications to a single platform.  There's going to be conflict there.

My only comment on that is this: Since this is a problem related to the installation of Python distributions, it should deal with the problems that Python developers have more forcefully than non-Python developers and non-programmers.

Thanks to venv, there's an alternative available that may be able to keep both of us happy: split the defaults. For system installs, adopt a vendor-centric, multi-language, easy-to-translate-to-language-neutral-packaging mindset (e.g. avoiding *.pth files by unpacking eggs to the file system). For venv installs, do whatever is most convenient for pure Python developers (e.g. leaving eggs packed and using *.pth files to extend sys.path within the venv).

One of Python's great virtues is its role as a glue language, and part of being an effective glue language is playing well with others. That should apply to packaging & distribution as well, not just to runtime bindings to tools written in other languages.

When we add the scientific users into the mix, we're actually getting to a third audience: multi-language developers that want to use Python's packaging utilities for their source and binary distribution formats.

The Python community covers a broad spectrum of use cases, and I suspect that's one of the big reasons packaging can get so contentious

Library authors: just want to quickly and easily publish their work on the Python package index in a way that is discoverable by others and allows feedback to reach them at their development site

Web developers: creators of Python applications, relying primarily on other Python software and underlying OS provided functionality, potentially with some native extensions, that may need to run on multiple platforms, but can require installation using a language specific mechanism by technical staff

Rich client developers: creators of Python applications relying primarily on other Python software and underlying OS provided functionality, potentially with native extensions, that need to run on multiple platforms, but must be installed using standard system utilities for the benefit of non-technical end users

Enterprise developers: creators of Python or mixed language applications that need to integrate with corporate system administration policies (including packaging, auditing and configuration management)

Scientists: creators of Python data analysis and modelling applications, with complex dependencies on software written in a variety of other languages and using various build systems

Python embedders: developers that embed a Python runtime inside a larger application

setuptools (or, perhaps, easyinstall, although I've seen enough posts about eggs being uploaded to PyPI to suspect otherwise), encourages the deployment of system configuration changes that alter the runtime environment of every single Python application executed on the system. That's simply not cool. Again, it would help if you tried it in anger.  What's the worst that could happen?  You might like it! ;-)

Oh, believe me, if I ever had distribution needs that required the power and flexibility of setuptools, I would reach for it in a heartbeat (in fact, I already use it today, albeit for tasks that ordinary distutils could probably handle). That said, I do get to cheat though - since I don't need to worry about cross-platform deployment, I can just use the relevant RPM hooks directly :)

You're right that most of my ire should be directed at the default behaviour of easy_install rather than at setuptools itself, though. I shall moderate my expressed opinions accordingly.

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list