[Python-ideas] Add a site.cfg to keep a persistent list of paths (original) (raw)

Paul Moore p.f.moore at gmail.com
Thu Oct 21 13:21:56 CEST 2010

On 21 October 2010 00:35, Floris Bruynooghe <flub at devork.be> wrote:

I've read your and Ian's responses and still don't understand what setup.py develop brings to the party which can't be done with simple PYTHONPATH.

I'm glad it's not just me!

Again, apologies if I understand the problem wrongly.  But I too am worried about too many complexities and "magic".  One of my main issues with setuptools is that it tries to handle my python environment (sys.path) outside of normally expected python mechanisms by modifying various custom files.  I would hate to see distutils2 repeat this.

I think that the key issue here is that PEP 376 introduces the idea of a "distribution" which is a somewhat vaguely defined concept, which can contain one or more packages or modules. Distributions don't have a well-defined directory structure, and don't participate properly in Python's standard import mechanism (PEP 302, PYTHONPATH, all that stuff). The distribution metadata (dist-info directory) is not package-based, and so doesn't fit the model.

Suggestions:

  1. PEP 376 clearly defines what a "distribution" (installed or otherwise) is, in terms of directory structure, whether/how it supports PEP302-style non-filesystem access, etc. I don't see a reason here why we can't mandate some structure, rather than leaving things as a "free for all" like the current setuptools/adhoc approach.
  2. Mechanisms for dealing with distributions are only discussed in terms of the PEP 376 definitions, so we have a common understanding.

As a first cut, I'd say that a distribution is defined purely in terms of its metadata (dist-info directory). On that basis, there should be a definition of where dist-info directories are searched for, PEP 376 seems to state that this is only in site-packages ("This PEP proposes an installation format inspired by one of the options in the EggFormats standard, the one that uses a distinct directory located in the site-packages directory."). And yet, this whole "develop" discussion seems to be about locating dist-info directories located elsewhere.

Having said that, PEP 376 later states:

get_distributions() -> iterator of Distribution instances. Provides an iterator that looks for .dist-info directories in sys.path and returns Distribution instances for each one of them.

This implies dist-info directories are searched for in sys.path. OK, fine. That's broader than just site-packages, but still well-defined and acceptable. And that's where I get my expectations that manipulating PYTHONPATH should work.

So what's this directory structure we're talking about with Foo containing two packages, and Foo.dist-info being alongside Foo? Foo itself isn't on PYTHONPATH, so why should Foo.dist-info be found at all? Based on PEP 376, it's not meant to be found.

Maybe if this is a requirement, it needs a change to PEP 376, which I guess means the PEP discussion and approval process needs to be gone through again. I, for one, would be OK with that, as I remain to be convinced that the complexity and confusion is worth it.

Paul.

More information about the Python-ideas mailing list