[Python-checkins] [Python-Dev] setuptools in the stdlib ( r45510 (original) (raw)

[Python-checkins] [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)

M.-A. Lemburg mal at egenix.com
Tue Apr 18 19:15:19 CEST 2006

Phillip J. Eby wrote:

At 10:55 AM 4/18/2006 +0200, M.-A. Lemburg wrote:

Phillip.eby wrote: > Author: phillip.eby > Date: Tue Apr 18 02:59:55 2006 > New Revision: 45510 > > Modified: > python/trunk/Lib/pkgutil.py > python/trunk/Lib/pydoc.py > Log: > Second phase of refactoring for runpy, pkgutil, pydoc, and setuptools > to share common PEP 302 support code, as described here: > > http://mail.python.org/pipermail/python-dev/2006-April/063724.html

Shouldn't this new module be named "pkglib" to be in line with the naming scheme used for all the other utility modules, e.g. httplib, imaplib, poplib, etc. ? It's not a new module; it was previously a module with only one function in it, introduced in Python 2.3. If it were a new module, I'd have inquired about a name for it first.

Didn't realize that. Too bad the time machine didn't work on this one :-/

> pydoc now supports PEP 302 importers, by way of utility functions in > pkgutil, such as 'walkpackages()'. It will properly document > modules that are in zip files, and is backward compatible to Python > 2.3 (setuptools installs for Python <2.5 will bundle it so pydoc_ _> doesn't break when used with eggs.)

Are you saying that the installation of setuptools in Python 2.3 and 2.4 will then overwrite the standard pydoc included with those versions ? Yes. As far as I can tell, there were no API changes to pydoc during this time, so this is effectively a "hot fix".

Why should a 3rd party extension be hot-fixing the standard Python distribution ?

This hot-fixing doesn't apply to setuptools system packages built with --root or --single-version-externally-managed, however, so OS vendors who build packages that wrap setuptools will have to make an explicit decision whether to also apply any fixes. If they do not, an end-user can of course install setuptools in their local PYTHONPATH and the hotfix will take effect.

What does setuptools have to do with pydoc ? Why should a user installing setuptools assume that some arbitrary stdlib modules get (effectively) replaced by installing setuptools ?

If you want to provide a hot fix for Python 2.3 and 2.4, you should make this a separate install, so that users are aware that their Python distribution is about to get modified in ways that have nothing to do with setuptools.

I bothered by the fact that installing setuptools actually changes the standard Python installation by either overriding stdlib modules or monkey-patching them at setuptools import time. Please feel free to propose alternative solutions that will still ensure that setuptools "just works" for end-users. Both this and the pydoc hotfix are "practicality beats purity" issues.

Not really. I'd consider them design flaws.

distutils is built to be extended without having to monkey-patch it, e.g. you can easily override commands with your own variants by supplying them via the cmdclass and distclass keyword arguments to setup().

By monkey patching distutils during import of setuptools, you effectively change distutils at run-time and not only for the application space that you implement in setuptools, but for all the rest of the application.

If an application wants to use setuptools for e.g. plugin management, then standard distutils features will get replaced by setuptools implementations which are not compatible to the standard distutils commands, effectively making it impossible to access the original versions.

Monkey patching is only a last resort in case nothing else works. In this case, it's not needed, since distutils provides the interfaces needed to extend its command classes via the setup() call.

See e.g. mxSetup.py in the eGenix extensions for an example of how effective the distutils design can be put to use without having to introduce lots of unwanted side-effects.

Add setuptools to the stdlib ? I'm still missing the PEP for this along with the needed discussion touching among other things, the change of the distutils standard "python setup.py install" to install an egg instead of a site package. Setuptools in the stdlib simply means that people wanting to use it can import it. It does not affect programs that do not import it. It also means that "python -m easyinstall" is available without having to first download ezsetup.py.

Doesn't really seem like much of an argument for the addition... the above is true for any 3rd party module.

As for discussion, Guido originally brought up the question here a few months ago, and it's been listed in PEP 356 for a while. I've also posted things related to the inclusion both here and in distutils-sig.

I know, but the discussions haven't really helped much in getting the setuptools design compatible with standard distutils.

Unless that's being put in place, I'm -1 on the addition, due to the invasive nature of setuptools and its various side-effects on systems management.

Note that it only takes one single module in an application doing "import setuptools" for whatever reason, to have it modify distutils at run-time.

The PEP is needed to document these effects, discuss their merits/drawbacks and find solutions to the problems prior to creating a status quo that's hard to modify after the fact due to the large install base of the Python distribution.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Apr 18 2006)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Python-checkins mailing list