[Python-Dev] Internal documentation for egg formats now available (original) (raw)

Phillip J. Eby pje at telecommunity.com
Fri Apr 28 19:51:32 CEST 2006

(Thank you, by the way, for actually reading some of the documentation before writing this post, and for asking questions instead of jumping to conclusions.)

At 06:43 PM 4/28/2006 +0200, M.-A. Lemburg wrote:

I've now found this section in the documentation which seems to have the reason:

http://peak.telecommunity.com/DevCenter/EasyInstall#compressed-installation Apart from the statement "because Python processes zipfile entries on sys.path much faster than it does directories." being wrong,

Measure it. Be sure to put the directories or zipfiles first on the path, not last. The easiest way to do so accurately would be to easy_install some packages first using --zip-ok and then using --always-unzip, and compare the two.

Finally, try installing them --multi-version, and compare that, to get the speed when none of the packages are explicitly put on sys.path.

it looks like all you'd have to do, is make --always-unzip the default.

You mean, all that you'd have to do is put it in your distutils configuration to make it the default for you, if for some reason you have a lot of programs that resemble "python -c 'pass'" in their import behavior. :)

Another nit which seems to have been introduced in 0.6a11: you now prepend egg directory entries to other sys.path entries, instead of appending them.

What's the reason for that ?

It eliminates the possibility of conflicts with system-installed packages, or packages installed using the distutils, and provides the ability to install stdlib upgrades.

Egg directory should really be treated just like any other site-package package and not be allowed to override stdlib modules and packages

Adding them after site-packages makes it impossible for a user to install a local upgrade to a system-installed package in site-packages.

One problem that I think you're not taking into consideration, is that setuptools does not overwrite packages except with an identical version. It thus cannot replace an existing "raw" installation of a package by the distutils, except by deleting it (which it used to support via --delete-conflicting) or by installing ahead of the conflict on sys.path.

Since one of the problems with using --delete-conflicting was that users often don't have write access to site-packages, it's far simpler to just organize sys.path so that eggs always take precedence over their parent directory. Thus, eggs in site-packages get precedence over site-packages itself, and eggs on PYTHONPATH get precedence over PYTHONPATH.

without explicit user action by e.g. adjusting PYTHONPATH.

Installing to a PYTHONPATH directory is an explicit user action. Installing something anywhere is an explicit user request: "I'd like this package to be importable, please." If you don't want what you install to be importable by default, use --multi-version, which installs packages but doesn't put them on sys.path until you ask for them at runtime.

More information about the Python-Dev mailing list