[Python-Dev] setuptools in 2.5. (original) (raw)

Phillip J. Eby pje at telecommunity.com
Fri Apr 21 01:14:11 CEST 2006

At 11:53 PM 4/20/2006 +0200, M.-A. Lemburg wrote:

Only after endless discussions, Phillip added some weird --long-option-name-that-no-one-can-remember to the setuptools "install" command that restores the standard distutils behavior which should be the default anyway.

And later, I also made it the default behavior for installation with --root, without any further prompting from you or anyone, in fact.

The suggestions that were made were constructive. Just look at the distutils-sig archive. Only very few of the suggestions were taken into account.

Actually, all of your suggestions were taken into account, by which I mean I have done all I can to understand them and to understand the concerns behind them -- as I do with all requests and feedback. Some, however, will be a long time coming in implementation, or are things I simply can't agree with. Those suggestions may or may not be "right" in some sense, but they are not right for setuptools.

But I take all suggestions into account, and sometimes I find ways to implement them. When I say "no", it simply means, "I don't see how I can do that without breaking some other constraint." It has happened repeatedly, however, that a way of doing something occurs to me months after the request, and then it is added with little fanfare.

Meanwhile, I'm always going to give a higher priority to the requests that are in strong alignment with the goals of setuptools, and come from:

It also helps if they are things I can actually come up with a way to implement, without violating setuptools' goals.

Instead, setuptools fans insisted on their right to have everything "just work" in their sense of the word.

Yeah, but they're not as bad as those Python fans with their crazy insistence on significant whitespace. ;)

Seriously, do you really think this is a bad thing?

Most of them probably don't even know what the distutils or Python standards are for installing packages, where the motivation to have zip imports came from and have only ever seen and used the setuptools way.

Um, yeah, because Jim Fulton is such a newbie to Python, surely he's never had to use Makefile.pre.in to build exotic C exten... Oh, wait, never mind. :)

The two camps need to come together again, but that will require understanding some of the history and standards that we've had in Python ever since Python packages were introduced. It will also require the setuptools folks to get some feeling of respect for those who have worked in the field for years.

I'm sorry, but this is an example of the highly patronizing attitude you've been giving on the distutils-sig since day one. As it happens, I too remember the world before distutils.

With due respect, Marc, not every Python developer is you or Fredrik or Jim. There are plenty of people with things to contribute to Python who find the distutils upsetting, confusing, or worse. When I started doing work for OSAF, I was very surprised to find out how many developers new to Python found the distutils not merely inconvenient (which was my personal perspective) but simply unusable. Some let forth streams of profanity at the mere mention of the distutils.

So, please consider this: to experts such as you and I, the distutils may merely be inconvenient or tedious, something to throw together an mxSetup.py or zpkgutil or some such convenient utilities to handle all your personal distribution needs and avoid the repetition and tedium. And in fact, setuptools originally began as just such a few extensions for my personal use.

But to Python newbies, the distutils is a confusing, broken, undocumented, arcane, archaic, unpredictable useless piece of ****, an embarrassment to Python, far worse than Java, and many many other things. And yes, those are all things that people have actually said to me. They really, really want things to Just Work, and they say so in so many words.

So when I work on setuptools, I imagine those developers, and I try to imagine what they will think and what they will expect.

To be frank, you haven't shown in our discussions that you respect or value those developers' perspective. You seem to believe that there are other things more important than making things Just Work for this audience. I don't know what those things are, but all I can say is, to me they don't exist. I just can't see it. Because all I can see is the frustration on the faces of developers who were enthusiastic about Python until they encountered the distutils.

Distutils is great. It's also unusable for developers who have better things to do with their time. The two are not mutually exclusive, as Fredrik has often enough pointed out regarding other problems with Python's "public face".

You may not know it, but having worked with distutils for a long time, I have great respect for Phillip's work - it's just that I find some of his design decisions wrong, since they don't follow the standards.

And that's where we disagree - standards exist to serve users, not the other way 'round. Standards change when times change. The choice to follow a standard is an economic decision for me, not a religious one. Where distutils works for setuptools' goals, I embraced it, and where it does not, I extended it. Call me Bill Gates if you must, but it is the reason that setuptools is successful, not blogosphere hype (you'll find almost as many anti-setuptools blog articles as pro- ones; the majority are actually just neutral) or screencasts and pundits (there are none).

It has "fans" because its design choices are strictly Darwinian: whatever makes setuptools be adopted by more packages is the right choice for setuptools. Any other considerations have been secondary. As Guido pointed out earlier, "acceptance is critical" for a system like this. That's why the rallying cry is that things should "Just Work" -- because that's what people want.

There's really nothing wrong with the standard distutils two step process:

1. download and unzip the source file 2. run "python setup.py install"

First, that's three steps. "Download and unzip" doesn't come in one-step form so far as I know.

Second, you're leaving out the part where you upgrade a package that renames or moves or removes a module. Your response on the distutils-sig was that the package author should write a distutils extension to do the necessary cleanup, which again highlights a complete lack of respect for package authors' time.

Not everyone who can contribute something valuable to Python's library base is willing to or even can become a distutils expert. Simple use cases like upgrading or even uninstalling a library should be simple. You should not have to be a packaging expert to share your package with the world.

I'm not saying setuptools is perfect for this either -- hence the 0.6 version number for the stable release. But it's the only thing that's even trying.

I've been publishing the mx Tools since 1997. Back then we only had the old Makefile.pre.in mechanism and still most people managed to get the tools working (step 2 then being: "make -f Makefile.pre.in boot; make; make install") without having to ask for help. Since I've started using distutils in 2001, the number of support questions related to compiling and installing the tools dropped to near zero.

I think this is quite a success story for distutils. Unfortunately, this fact is rarely mentioned in all these setuptools discussions, probably because it's just not the latest greatest and shiniest tool in the world anymore as it was perceived in the early days.

I wish people would stop interpreting setuptools as some kind of a dig at the distutils. Does the existence of Python 2.5 mean that Python 2.4 is bad? Does the existence of third-party libraries make the standard library incomplete?

More information about the Python-Dev mailing list