[Python-Dev] Distutils ML wrap-up: setup.cfg new format (original) (raw)

Tarek Ziadé ziade.tarek at gmail.com
Tue Sep 22 15:21:06 CEST 2009

Hello

Here's a wrapup of the Distutils-SIG discussion we had on the "static metadata" topic.

I realize that it's a good thing to send in. python-dev such wrapup on distutils design decisions, to keep everyone informed and get more feedback when required.

I will try to do it for every upcoming changes that are not going in a PEP process (when it's not a 'big' change). The rate of such mails should not be very high. (around 1 mail in python-dev for +150 mails in distutils-SIG)

If you feel that what we are about to change in distutils is wrong, you can go ahead and help us by participating in Distutils-ML, so we keep one and only one media for these discussions.

The four sentences summary for people in a hurry:

Getting metadata of a distribution that is not.
installed means running its setup.py. This means.
downloading the whole archive, and running.
third party code on your system.

To avoid it, we are adding a [setup] section in.
setup.cfg where you can express the package metadata
statically.

Conditional sections, specific to some system.
can be added to add some specific fields in [setup].

At the end, you will be able to get metadata fields
without running any third-party code, and possibly
get only the distribution setup.cfg for this.

Now the long version.

Rational

Today, if you want to list all the metadata (PEP 314) of a. distribution that is not installed, you need to use it's. setup.py CLI.

So, basically, you download it, and run::

$ python setup.py --name
Foo

$ python setup.py --version
1.2

Where name and version are metadata fields. That's OK but as. soon as the developers add more code in setup.py, this feature might break or (worse) might do unwanted things on the target. system.

Why should we run third-party code just to get its metadata ?

So we worked on a way to express those metadata statically, by pushing them in setup.cfg. That's not hard, since all. the metadata fields are static values.

Adding a setup section in setup.cfg

So the first thing we want to introduce is a [setup] section, that may contain any field from the metadata::

[setup]
name: Foo
version: 1.2

Distutils will look for them, and will use them. Of course setup.py is still required, and in case an option that is. a metadata field is passed to setup(), it will override one. located in setup.cfg.

PEP 341 is coming up

Remember the last Pycon Summit ? We said that we would introduce a new metadata field to describe requirements.. That's basically what PEP 341 is about, and we are still. working on it.

Basically, we will be able to write things like::

requires: Twisted == 8.2.0

What takes us so long is that adding requirements like this in the metadata requires more things:

And the later makes our "setup.cfg" new [setup] section. obsolete as soon as this new metadata field is added in. Python.

So we need more that what I've shown in the previous section

Context-dependant sections

The second change we are going to introduce is context-dependant sections in setup.cfg.

A context dependant section is a section with a condition that is. used only if the execution environment meets its condition.

Here's an example::

[setup]
name: Foo
version: 1.3

[setup:sys_platform == 'win32']
requires: pywin32
requires: bar > 1.0

[setup:os_machine == '64bits']
requires: some_package

[setup:python_version == '2.4' or python_version == '2.5']
requires: some_package

Every [setup:condition] section will be added in [setup] only if the condition is met.

The micro-language behind this is the simplest possible: it compares only strings, with usual string operators, and with the ability to combine expressions. It makes it also easy to understand to non-pythoneers (os packagers).

The pseudo-grammar is (I don't know how to write those but you'll get it hopefully)::

comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'
comparison: expr (comp_op expr)*
expr: STRING
test: or_test
or_test: and_test ('or' and_test)*
and_test: not_test ('and' not_test)*
not_test: 'not' not_test | comparison

where STRING belongs to any of those:

Distutils will provide a function that is able to read the metadata of a distribution, given a setup.cfg file, for the target environment::

>>> from distutils.util import local_metadata
>>> local_metadata('setup.cfg')
<DistributionMetadata instance>

Meaning that a Vanilla Python will be able to read the metadata of a package without running any code.

That's what distutils will use when setup.py is run.

That's what any package manager will be able to use to work. with distributions.

Limitations

If a distribution is unable for obscure reasons (or does not wish) to set all metadata fields in setup.cfg, that's fine :. the fields will be set to UNKNOWN when local_metadata is called.

When setup.py is run, options passed to setup() will complete those.

When a package manager gets 'UNKOWN' values, it knows it might need to do further processing by running setup.py.

Why this change is good

Once the requires metadata is added in PEP 341, being able to query the metadata for a distribution can be done without doing anything else than reading a static file and interpreting its conditions in a restricted and secured manner.

So there's no need to run third-party code anymore.

Possible use cases:

And, again, this new [setup] section is backward-compatible, and will be a good complement to the work done in PEP 376.

Thanks for reading so far. This code will be added in 2.7 and 3.2.

Tarek

-- Tarek Ziadé | http://ziade.org | オープンソースはすごい!

More information about the Python-Dev mailing list