[Python-Dev] [Distutils] PEP 426 is now the draft spec for distribution metadata 2.0 (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Tue Feb 19 12:42:20 CET 2013

On Tue, Feb 19, 2013 at 8:36 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

Le Tue, 19 Feb 2013 10:24:15 +0000, Paul Moore <p.f.moore at gmail.com> a écrit :

On Tuesday, 19 February 2013, M.-A. Lemburg wrote:

> > The only tool in wide spread use that understands part of the 1.2 > data is setuptools/distribute, but it can only understand the > Requires-Dist field of that version of the spec (only because the > 1.1 Requires field was deprecated) and interprets a Provides-Extra > field which isn't even standard. All other 1.2 fields are ignored. > setuptools/distribute still writes 1.1 meta-data. > > I've never seen environment markers being used or supported > in the wild. > > I'm not against modernizing the format, but given that version 1.2 > has been out for around 8 years now, without much following, > I think we need to make the implementation bit a requirement > before accepting the PEP. > The wheel project uses metadata 2.0 and environment markers - indeed, the PEP was written to formalise what wheel was implementing (specifically so that pip was happy to incorporate support). I'm unsure what this means. Does the "wheel project" (I thought wheel was a format? is it also a project?) reimplement its own wheel of metadata parsing and generating? I agree that standard library support would be good, either via Distutils or by incorporating distlib, but I don't think it is essential for acceptance. I think Marc-André is right that the acceptability of the standard should be judged on the availability of (preferably standard) implementations. If the standard isn't implemented, then perhaps it means it's too ambitious / too complex / victim of the second-system effect.

The most complex part of the PEP is still the version ordering, and that's implemented in the analysis script (initially by Vinay, with one small tweak by me to fix the regular expression for the new validity rules). Everything else was either already implemented for distutils2/distlib, or is a trivial check (like scanning for "a", "b", "c" or "dev" to identify pre-releases and filter them out if none of those characters are present in the corresponding version specifier).

Nothing in the PEP is particularly original - almost all of it is either stolen from other build and packaging systems, or is designed to provide a discoverable alternative to existing setuptools/distribute/pip practices (specifically, the new extension mechanism means things like entry points can be defined in the standard metadata files without python-dev needing to get involved).

Probably the most novel concepts in the PEP are "Obsoleted-By" (so a project can issue one last release to inform users directly that it is dead, and there is an alternative available) and the "Private-Version" field to handle the case where a projects' preferred version scheme requires translation to match the more restricted format understood by the installation tools.

Yes, the spec in aggregate is complex, but it's complex because software distribution and dependency analysis are complex tasks. If people want to simplify it, they can try to rebut the rationale for the individual changes that is included at the end of the PEP, but "it's complex" alone is not an argument.

So I kind of agree with him the PEP shouldn't be accepted until there's a decent patch pending for the stdlib. That's not a judgement on the PEP's quality. Just an advice to remain cautious, especially if the previous metadata version already enjoys a very poor adoption rate.

There were at least a couple of very good reasons for not adopting metadata 1.2:

The adoption of metadata 1.2 also got caught up in the "distutils must die" rhetoric that meant the distutils2 project got bogged down in trying to eliminate setup.py completely, when the only thing that really needs to die is "./setup.py install" (and even that can be acceptable on a development system, we just want to eliminate the need to ever run it on a deployment target). That's the key change in philosophy between this PEP and the previous one: I'm working backwards from what installers need, and creating a specification that allows build tools, including distutils, to give it to them.

I don't really care how those build tools get the info from their users - setup.py, setup.cfg, bento.info, it doesn't matter to me.

By the way, pip may be nice, but it's still a third-party tool, and perhaps not even as widespread as distribute / setuptools.

The main easy_install feature that pip couldn't previously match was installing from pre-built binaries (since easy_install had eggs), which is why the wheel format is so important.

My current plan for Python 3.4 is for pip to be formally endorsed as the recommended tool that supports installing from an sdist, while the stdlib will ship a "pydist" tool that can only install from prebuilt wheel files. This will keep the stdlib tool simple (i.e. no need to try to find installers on arbitrary target systems), while easily allowing users to bootstrap a full fledged installer through "pydist install pip". (This will all play it out in PEPs before 3.4 rolls around - and there will likely be recruiting and documentation efforts well in advance of that)

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list