[Python-Dev] Internal documentation for egg formats now available (original) (raw)

Phillip J. Eby pje at telecommunity.com
Thu Apr 27 19:52:02 CEST 2006

At 06:47 PM 4/27/2006 +0200, M.-A. Lemburg wrote:

Just read that you are hijacking site.py for setuptools' "just works" purposes.

"hijacking" isn't the word I'd use; "wrapping" is what it actually does. The standard site.py is executed, there is just some pre- and post-processing of sys.path.

Please be aware that by allowing .pth files in all PYTHONPATH directories you are opening up a security hole - anyone with write-permission to one of these .pth files could manipulate other user's use of Python.

FUD. Write access to a PYTHONPATH-listed directory already implies complete control over the user's Python environment. This doesn't introduce any issues that aren't implicit in the very existence of PYTHONPATH.

That's the reason why only site-packages .pth files are taken into account, since normally only root has write access to this directory.

False. On OS X, Python processes any .pth files that are found in the ~/Library/Python2.x/site-packages directory. (Which means, by the way, that OS X users don't need most of these hacks; they just point their install directory to their personal site-packages, and it already Just Works. Setuptools only introduces PYTHONPATH hacks to make this work on other platforms.)

The added startup time for scanning PYTHONPATH for .pth files and processing them is also not mentioned in the documentation. Every Python invocation would have to pay for this - regardless of whether eggs are used or not.

FUD again. This happens if and only if:

  1. You used easy_install to install a package in such a way that the .pth file is required (i.e., you don't choose multi-version mode or single-version externally-managed mode)

  2. You include the affected directory in PYTHONPATH

So the idea that "every Python invocation would have to pay for this" is false. People who care about the performance have plenty of options for controlling this.

Is there a nice HOWTO that explains what to do if you care more about performance than convenience? No. Feel free to contribute one.

I really wish that we could agree on an installation format for package (meta) data that does not rely on ZIP files.

There is one already, and it's used if you select single-version externally-managed mode explicitly, or if you install using --root.

All the unnecessary magic that you have in your design would just go away - together with most of the issues people on this list have with it.

Would you settle for a way to make a one-time ~/.pydistutils.cfg entry that will make setuptools act the way you want it to? That is, a way to make setuptools-based packages default to --single-version-externally-managed mode for installation on a given machine or machine(s)? That way, you'll never have to wonder whether a package uses setuptools or not, you can just "setup.py install" and be happy.

More information about the Python-Dev mailing list