[Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py) (original) (raw)

Phillip J. Eby pje at telecommunity.com
Wed Apr 19 01:59:36 CEST 2006

At 12:49 AM 4/19/2006 +0200, Martin v. Löwis wrote:

Fredrik Lundh wrote: > it's still listed under "possible additions" in the release PEP, and there don't > seem to be a PEP or any other easily located document that explains exactly > how it works, what's required from library developers, how it affects existing > toolchains, etc. is this really ready for inclusion ? does anyone but Phillip > understand how it works ?

I don't understand it.

Have you read the manuals?

My concern is that it appears to involve a lot of magic.

Please define "magic". Better, please point to the API functions or classes, or the setup commands documented in the manual, to show me what these things are that appear to be magic.

There do exist implementation details that are not included in user documentation, but this is going to be true of any system. These details are sometimes necessarily complex due to distutils limitations, behavioral quirks of deployed packages using distutils, and the sometimes baroque variations in sys.path resolution across platforms, Python versions, and invocation methods.

That these details are not discussed at length in the user documentation is because they are boring hideous things that I wish I didn't have to know myself, and that few people using setuptools would ever want to know.

They do, however, get discussed (or at least explained by me) at length on the distutils-sig. I explain the quirks and the tradeoffs for each bit of implementation in detail there, and have been doing so for nearly a year now. Anybody who wants to know what's going on has had plenty of opportunity to learn.

This magic might do the "right thing" in many cases, and it might indeed help the user that the magic is present, yet I still do believe that magic is inherently evil: explicit is better than implicit.

Are documented defaults "implicit" or "magic"?

To the extent that there is anything that may be called "magic" in setuptools, it exists only because the necessary infrastructure isn't already present in Python, or because it was required to work around the quirks of other systems. Setuptools chooses to "work at all costs" because backward-compatibility skirts the chicken-and-egg problems that would otherwise exist.

If setuptools weren't so highly backward-compatible, its use would never have caught on enough to allow this discussion to be taking place in the first place. :)

The primary place where it isn't backward compatible is if somebody does "setup.py install" without using --root, and they care about the exact file layout of the result, and they are installing a package that explicitly uses setuptools... in which case their complaint is with the author of the package, if the author didn't explain that they used setuptools or point them to relevant documentation.

Setuptools' manual prominently explains what developers should tell their users, if they use setuptools in their setup.py:

"""To keep these users happy, you should review the following topics in your project's installation instructions, if they are relevant to your project and your target audience isn't already familiar with setuptools and easy_install."""

http://peak.telecommunity.com/DevCenter/setuptools#what-your-users-should-know

With respect to you and MAL, I think that some of your judgments regarding setuptools may have perhaps been largely formed at a time last year when, among other things:

These are significant changes that are directly relevant to the objections that you and he raised (regarding startup time, path length, tools compatibility, etc.), and which I added because of those objections. I think that they are appropriate responses to the issues raised, in that they allow the audience that cares about them (you, MAL, and system packagers in general) to avoid the problems that those features' absence caused.

It would probably helpful if you would both be as specific as possible in your objections so that they can be addressed individually. If you don't want setuptools in 2.5, let's be clear either on the specific objections. If the objection is to the very idea of setuptools, that's fine too, as long as we're clear that's what the objection is.

So I would ask that you please make a list of what you would have to see changed in setuptools before you would consider it viable for stdlib inclusion, or simply admit that there are no changes that would satisfy you, or that you don't know enough about it to say, or that you'd like it to be kicked back to distutils-sig for more discussion ad infinitum, or whatever your actual objections are.

Then I will make my responses to your proposals, and then Guido can have his say based on the cons from you and the pro's from me. If he says no for 2.5, that's okay by me. I didn't propose its inclusion, the users (and Guido) did. But now that I've busted butt to get it ready in time, I'd prefer that any withdrawal decision be based on actual facts, rather than FUD, hand-waving, and vague innuendo. Meanwhile, this discussion has used up the time that I otherwise would have spent writing 2.5 documentation today (e.g., for the pkgutil additions).

More information about the Python-Dev mailing list