[Python-Dev] Internal documentation for egg formats now available (original) (raw)

Phillip J. Eby pje at telecommunity.com
Fri Apr 28 19:33:51 CEST 2006

At 06:20 PM 4/28/2006 +0200, M.-A. Lemburg wrote:

A single malicious .pth file on PYTHONPATH could trick the user into running Python modules that he isn't really aware of.

Adding directories to PYTHONPATH which are writable by others is easy, just add '/tmp' as entry in some .pth file. Then put e.g. a modified os.py into /tmp and have it execute 'rm -rf /' at import time.

You still haven't explained how these malicious .pth files or entries come into being without the user's permission. How does this malicious user get write access to the .pth file to put /tmp into it? Whatever your answer is, that is the security hole, not the existence of usable .pth files.

So this is still just handwaving and FUD.

> And if you replace 'syspathtools.use()' in that email, with > 'pkgresources.require()', then it describes *exactly how setuptools > works with .egg directories today*.

Interesting that you used that idea, because back then you didn't reply to the email. Looks like I deserve some credit ;-)

In truth, I completely forgot about that post and only rediscovered it yesterday. When Bob and I were doing the initial design, he said he thought that the naively-unzipped form of an egg should be usable as-is, and I agreed that it made sense to do that.

If you've already implemented this (which I wasn't aware of, since when I played with setuptools it kept installing .egg ZIP files),

See the --always-unzip option to easy_install, which has been there since 0.5a9 -- i.e., since July 2005. Package authors can also mark their packages "zip_safe=False" to force it to be installed unzipped (unless the user forces it to be zipped using --zip-ok).

then why don't you make .egg directories the standard installation scheme, instead of insisting on having .egg ZIP files in site-packages/ ?

Because zip files add less import overhead over the life of a program (even a relatively short program) than directories do!

(Which, by the way, I believe I've explained to you on the distutils-SIG on more than one occasion, during some of the previous N times you brought this up.)

I wonder why you don't always create .egg directories.

Performance, plain and simple.

What you should be benchmarking when you test performance impact, is twenty directories at the front of sys.path vs. twenty zipfiles at the front, both for "python -c pass" and for a program that does a few stdlib imports. That will show you why EasyInstall never unzips eggs if it can help it.

* I am against using ZIP as medium for package installation.

Then add this to your distutils.cfg or ~/.pydistutils.cfg and be happy forevermore:

[easy_install] always_unzip = 1

If, however, you mean that you want everyone else to do as you prefer, forget about it, 'cause it's not going to happen. You aren't setuptools' target user, so the defaults aren't going to be catering to your prejudices -- especially not the ones based on ignorance, like this one.

* I am against using standard distutils commands to mean different things depending on which set of packages you have installed (in this case the "install" command)

And as I just pointed out, you've failed to show that there is any practical purpose to this. I've already made the behavior an option, and it's the default behavior when you use --root. You haven't shown any actual use case for it being the default behavior, and practicality beats purity. What's more, you said you don't want the option to make it the default behavior for you. Which means that this is only about you deciding what is "right" for other people than you to have, so I don't see any reason to consider that subject any further. If other people want that behavior, they can certainly speak up for themselves about what actual (not hypothetical) problems it creates for them. (And when they do speak up, as some system packagers did, I made --root enable the old behavior and that addressed the actual problems the new behavior created for that audience.)

Phillip, you're always side-stepping answering questions (and cutting away quoted email text doesn't help either).

Repeating arguments I've already shown to be spurious helps even less. I don't see much point to chasing red herrings and straw men, either.

You should be aware that both package authors and users are capable of learning and adapting.

Then they can certainly adapt to "install" doing the right thing for most users and most packages: to install things in such a way that they can be uninstalled or upgraded cleanly, by default.

If we tell package authors to put a link to their eggs on PyPI along with an MD5 sum, then the users be in a much better position and could feel more secure about what they download and install.

People who want to do that can easily use:

easy_install --allow-hosts=*.python.org

or put it in their configuration files to make it the default behavior. See:

http://peak.telecommunity.com/DevCenter/EasyInstall#restricting-downloads-with-allow-hosts

Have a nice weekend,

It'd be an even nicer one if people objecting to things would read the documentation first, to find out whether their objections actually apply. Your zip vs. directory arguments would've been moot if you'd read things like this:

http://peak.telecommunity.com/DevCenter/EasyInstall#compressed-installation

Or for that matter, if you'd read the documentation that was linked to in the post that started this thread, which explains that .egg can be a zipfile or a directory. Or if you'd read the parts of the EasyInstall documentation where it explains that OS X already processes .pth files from ~/Library/Python2.x/site-packages.

If you don't have time to read the documentation, please have the courtesy to ask questions to try to alleviate your concerns, preferably before you launch attacks using nothing but vague premonitions based on ignorance.

More information about the Python-Dev mailing list