[Python-Dev] setuptools in 2.5. (summary) (original) (raw)

M.-A. Lemburg mal at egenix.com
Thu Apr 20 15:47:47 CEST 2006

• Previous message: [Python-Dev] setuptools in 2.5.
• Next message: [Python-Dev] setuptools in 2.5. (summary)
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Anthony Baxter wrote:

In an attempt to help this thread reach some sort of resolution, here's a collection of arguments against and in favour of setuptools in 2.5. My conclusions are at the end.

Thanks for the summary. I'd like to add some important aspects (for me at least) that are missing:

• setuptools should not change the standard distutils "install" command to install everything as eggs

  Eggs are just one distribution format out of many. They do server their purpose, just like RPMs, DEBs or Windows installers do.

  However, when running "python setup.py install" you are in fact installing from source, so there's no need to wrap things up again.

  The distutils default of actually installing things in the standard Python is good, has worked for years and should continue to do so.

  The extra information needed by the dependency checking can easily be added to the package directory of the installed package or stored elsewhere in a repository of installed packages or as separate egg-info directory if we want to stick with setuptools' way of using the path name for getting meta-information on a package.

  Placing the egg-files into the system as ZIP files should be an option, e.g. as separate "install_egg" command, not the default.

• setuptools includes many good additions to distutils

  We should remember that setuptools will get part of the stdlib, so instead of monkey-patching distutils, a lot of setuptools features can be ported to distutils directly.

  For things that don't fit distutils, proper hooks should be added, so that extending distutils in the setuptools way can be done by the standard distutils extension mechanisms.

  I think that if Phillip were more open to reconsidering design decisions in setuptools like the decision to replace the install command instead of just adding a new install_egg command (together with a bdist_egg), quite a lot of code could be directly imported into core distutils.

  Even for things like his alternative sdist implementation could go in, albeit using a different command name.

• arguments like "just works" simply don't count

  Why not ? Because "just works" is not well-defined. If an implementation makes too many magical decisions for the user, you end up with a tunnel - with only one way in and one way out.

  In reality there are too many things you'd need magical decisions for. A generic distribution mechanism will never be capable of making all the right decisions for a user. Adding endless lists of options doesn't make things easier either.

  Let's face it: there isn't the one right way to do software distribution.

  What's good about it: there doesn't need to be.

  The package authors will know which particular tweaks are needed to make installing their package a breeze and easy for the user. So the right way to get close the "just works" idea, is to leave a door open for module authors and packages to add such functionality.

  Freedom of choice is often enough a lot better than any variant of "just works". Explicit is better than implicit. Good defaults help.

• setuptools is not ready for prime-time

  This is not quite right. setuptools works and has been used by quite a few people in the past.

  However, just dumping setuptools into the stdlib is considered a less than ideal approach. Instead, setuptools should get integrated into the stdlib.

  This process will take longer than the current 2.5 release cycle. We should at least mark setuptools as it is now in the stdlib as experimental and leave open the possibility of making changes to it.

The arguments against:

- Someone should instead just fix distutils. Right. And the amount of yelling if distutils changed in a non-b/w compat way wouldn't be high. There's also the basic problem that the distutils code is horrible. - It monkeypatches pydoc and distutils! It only monkeypatches pydoc when the separate setuptools installer is used on older Pythons. How is this relevant for this discussion of Python 2.5? The monkeypatching for distutils should be reduced - see AMK's message for a breakdown of this. - Documentation beaker% pydoc xmlcore.etree no Python documentation found for 'xmlcore.etree' beaker% pydoc ctypes no Python documentation found for 'ctypes' The documentation (of which there is plenty) can and will be folded into the standard python docs. Most of the new modules in 2.5 went in before their docs. - Where's the PEP? I don't see the need. The stuff that could go into a PEP about formats and the like should go into the existing Distutils documentation. It's a far more useful place, and many more people are likely to find it there than in a PEP.

The PEP is needed to document the current discussion, not the setuptools technical documentation (this should of course go into the standard Python documentation).

Your summary along with the opinions of the ones who have commented it would be a good start. The PEP should also include the lists of things Martin and Fredrik have put forward.

Finally, the goal of the PEP should be to list the steps needed to integrate setuptools into the stdlib.

- It's a huge amount of code (or "ball of mud"), or, it adds too many features.

Most of these have been added over the last 2 years in response to feedback and requests from people on distutils-sig. There's been an obvious pent-up demand for a bunch of this work, and now that someone's working on it, these can get done. - It will break existing setup.py scripts! No it won't. If you don't type the letters 'import setuptools' into your setup.py, it won't be affected. - Rewriting from scratch is bad This isn't a rewrite - it's built on top of distutils. (An aside, I don't buy the "never rewrite" argument. As I mentioned in an earlier message, look at urllib2, twisted and email for starters. In addition, look at Firefox, Windows XP, and Mac OSX. Hell, Linux could be considered a rewrite of Minix, once upon a time.) - Eggs are inferior to distribution-specific packaging Not all operating systems have a decent packaging system. The ones that do, don't support multiple versions of the same library. In addition, there's no reason why existing packaging systems can't just bundle up the code as they do now - if they also add a .egg-info file to the packages, that would be even better! Finally, these don't support user installation of software. This is particularly useful in a hosting environment.

I think you're misrepresenting the argument here: eggs are not considered inferior. What packagers and application builders gripe about is the fact that setuptools forces them to use eggs instead of the standard Python way of installing packages.

I don't think anyone would be bothered by adding dependency information to their packages and thus make easy_install and friends work with regular Python packages as well.

And now let's look at some of the stuff that setuptools gives us:

- We have a CPAN-type system I do quite a number of Python talks, and this is always one of the most requested features. There's been many attempts to write this, none have been completed until now. If you honestly don't see that this is a big thing for Python, then I am very, very suprised. I suspect that this will be the #1 new feature of Python 2.5 that the users will notice and be happy about. - Multiple installs of different versions of the same package, including per-user installs. Again, as Python gets more widely used, this becomes a big issue. Sure, it's not necessarily a killer argument for python-dev, but stuff that's added to Python shouldn't just be just for the use of python-dev. The multiple installed versions feature also avoids the CPAN dependency hell problem - back when I used to work with Perl, this was a constant source of nightmarish problems.

I've never had a need for having multiple versions of a single package installed, so can't comment.

User installation is certainly possible with stock distutils.

The main problem with distutils is the lack of user documentation, not so much the lack of features.

- The "develop" mode

This makes life that bit less painful all-round.

Does it ? I usually point PYTHONPATH at my development tree and that's all it takes to do setup "develop" mode.

- The plugin/extension support

Extending distutils currently is a total pain in the arse.

Views differ on this one and most of this perception is due to missing distutils documentation.

- Backwards compatibility

easyinstall even works with existing packages that use traditional distutils, so long as they're in the Cheeseshop. Damn, this is nice. If you don't want to do the work to change your installation code, don't bother - it will still be useful. The conclusions: I'm a little suprised by the amount of fear and loathing this has generated. To me, there are such obvious benefits that I don't see why people are so vehemently against setuptools. I haven't seen any arguments that have convinced me that this isn't the right thing to do. Yes, there's still work to be done - but hell, we've only released the first alpha so far. For inclusion in the standard library, the usual benchmark is that the code offers useful functionality, and that it be the "best of breed". setuptools clearly meets these two criteria. (Ok, it's really "only of breed", but that also makes it "best", by default ). It's also been under development for over 2 years - according to svn, 0.0.1 was checked into svn back in March 2004. I'm also suprised by how much some people seem to think that the current state of distutils functionality is acceptable or desirable. It's not - it's a mess. Finally, I'd like to point out that I think some of the hostility towards Phillip's work has been excessive. He's done an amazing amount of work on this (look at the distutils-sig archive for the last two years for more), and produced something that's very very useful. He deserves far more credit for this than he seems to have been getting here.

He does and there's no doubt about it.

It would help a lot, though, if Phillip would be willing to reconsider some defaults he's using in setuptools to make them compatible to Python's default import mechanism.

A lot of the perceived hostility would go away if he'd play nice with standards that have existed in Python for years.

If Phillip thinks that we should change Python to always import packages from egg-files, then this would be a major change in the way Python works and thus require a complete PEP process.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Apr 20 2006)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

• Previous message: [Python-Dev] setuptools in 2.5.
• Next message: [Python-Dev] setuptools in 2.5. (summary)
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the Python-Dev mailing list