[Python-Dev] PEP 407 / splitting the stdlib (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Wed Jan 18 15:08:49 CET 2012

On Wed, Jan 18, 2012 at 10:30 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

Splitting the stdlib: - requires someone to do the splitting (highly non-trivial given the interactions of some modules with interpreter details or low-level C code) - requires setting up separate resources (continuous integration with N stdlib versions and M interpreter versions, for example) - requires separate maintenance and releases for the stdlib (but with non-trivial interaction with interpreter maintenance, since they will affect each other and must be synchronized for Python to be usable at all) - requires more attention by users since there are now two release schedules and independent version numbers to track

Did you read what I actually proposed? I specifically didn't propose separate stdlib releases (for all the reasons you point out), only separate date based stdlib versioning. Distribution of the CPython interpreter + stdlib would remain monolithic, as it is today. Any given stdlib release would only be supported for the most recent language release. The only difference is that between language releases, where we currently only release maintenance builds, we'd also release a second version of each maintenance build with an updated standard library, along with an alpha release of the next language version (with the last part being entirely optional, but I figured I may as well make the suggestion since I like the idea to encourage getting syntax updates and the like out for earlier experimentation).

When you initially pitched the proposal via email, you didn't include the "language moratarium applies to interim releases" idea. That one additional suggestion makes the whole concept much more appealing to me, but I only like it on the condition that we decouple the stdlib versioning from the language definition versioning (even though I recommend we only officially support very specific combinations of the two). My suggestion is really just a concrete proposal for implementing Ezio's idea of only bumping the Python version for releases with long term support, and using some other mechanism to distinguish the interim releases.

So, assuming a 2 year LTS cycle, the released versions up to February 2015 with my suggestion would end up being:

From the default branch: Python 3.3.0 + stdlib 12.08.0 (August 2012) Python 3.4.0a1 + stdlib 14.08.0a1 (February 2013) Python 3.4.0a2 + stdlib 14.08.0a2 (August 2013) Python 3.4.0a3 + stdlib 14.08.0a3 (February 2014) Python 3.4.0a4 + stdlib 14.08.0a4 (2014) Python 3.4.0b1 + stdlib 14.08.0b1 (2014) Python 3.4.0b2 + stdlib 14.08.0b2 (2014) Python 3.4.0c1 + stdlib 14.08.0c1 (2014) Python 3.4.0 + stdlib 14.08 (August 2014) Python 3.5.0a1 + stdlib 16.08.0a1 (February 2015)

From the 3.3 maintenance branch (these are maintenance updates to the "LTS" release): Python 3.3.1 + stdlib 12.08.1 (February 2013) Python 3.3.2 + stdlib 12.08.2 (August 2013) Python 3.3.3 + stdlib 12.08.3 (February 2014) Python 3.3.4 + stdlib 12.08.4 (August 2014) (and 3.3 branch enters security patch only mode)

From the 3.4 maintenance branch (these are maintenance updates to the "LTS" release): Python 3.4.1 + stdlib 14.08.1 (~February 2015)

From the stdlib feature development branch (these are the new interim releases with standard library updates only as proposed by PEP 407): Python 3.3.1 + stdlib 13.02.0 (February 2013) Python 3.3.2 + stdlib 13.08.0 (August 2013) Python 3.3.3 + stdlib 14.02.0 (February 2014) (only upgrade path from here is to make the jump to 3.4.0) -- 3.4.0 + 12.08.0 is released from default branch -- Python 3.4.1 + stdlib 15.02.0 (February 2015)

If we have to make "brown paper bag" releases for the maintenance or stdlib branches then the micro versions get bumped - the date based version of the standard library versions relates to when that particular API was realised, not when bugs were last fixed in it. If a target release date slips, then the stdlib version would be increased accordingly (cf. Ubuntu 6.06).

Yes, we'd have an extra set of active buildbots to handle the stdlib branch, but a) that's no harder than creating the buildbots for a new maintenance branch and b) the interim release proposal will need to separate language level changes from stdlib level changes anyway.

As far as how sys.version checks would be updated, I would propose a simple API addition to track the new date-based standard lib versioning: sys.stdlib_version. People could choose to just depend on a specific Python version (implicitly depending on the stdlib version that was originally shipped with that version of CPython), or they may instead decide to depend on a specific stdlib version (implicitly depending on the first Python version that was shipped with that stdlib).

The reason I like this scheme is that it allows us (and users) to precisely track the things that can vary at the two different rates. At least the following would still be governed by changes in the first two fields of sys.version (i.e. the major Python version): - deprecation policy - language syntax - compiler AST - C ABI stability - Windows compilation suite and C runtime version - anything else we decide to link with the Python language version (e.g. default pickle protocol)

However, the addition of date based stdlib versioning would allow us to clearly identify the new interim releases proposed by PEP 407 without mucking up all those things that are currently linked to sys.version and really shouldn't be getting updated every 6 months. Users get a clear guarantee that if they follow the stdlib updates instead of the regular maintenance releases, they'll get nice new features along with their bug fixes, but no new deprecations or backwards incompatible API changes. However, they're also going to be obliged to transition to each new language release as it comes out if they want to continue getting security updates.

Basically, what it boils down to is that I'm now +1 on the general proposal in the PEP, so long as:

  1. We get a separate Hg branch for "stdlib only" changes and default becomes the destination specifically for "language update" changes (with the latter being a superset of the former)
  2. The proposed "interim releases" are denoted by a new date-based sys.stdlib_version field and sys.version retains its current meaning (and slow rate of change)

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list