[Python-Dev] PEP 407 / splitting the stdlib (original) (raw)

Brett Cannon brett at python.org
Wed Jan 18 18:56:21 CET 2012

On Wed, Jan 18, 2012 at 09:08, Nick Coghlan <ncoghlan at gmail.com> wrote:

On Wed, Jan 18, 2012 at 10:30 PM, Antoine Pitrou <solipsis at pitrou.net> wrote: > Splitting the stdlib: > - requires someone to do the splitting (highly non-trivial given the > interactions of some modules with interpreter details or low-level C > code) > - requires setting up separate resources (continuous integration with N > stdlib versions and M interpreter versions, for example) > - requires separate maintenance and releases for the stdlib (but with > non-trivial interaction with interpreter maintenance, since they will > affect each other and must be synchronized for Python to be usable at > all) > - requires more attention by users since there are now two release > schedules and independent version numbers to track

Did you read what I actually proposed? I specifically didn't propose separate stdlib releases (for all the reasons you point out), only separate date based stdlib versioning. Distribution of the CPython interpreter + stdlib would remain monolithic, as it is today. Any given stdlib release would only be supported for the most recent language release. The only difference is that between language releases, where we currently only release maintenance builds, we'd also release a second version of each maintenance build with an updated standard library, along with an alpha release of the next language version (with the last part being entirely optional, but I figured I may as well make the suggestion since I like the idea to encourage getting syntax updates and the like out for earlier experimentation).

When you initially pitched the proposal via email, you didn't include the "language moratarium applies to interim releases" idea. That one additional suggestion makes the whole concept much more appealing to me, but I only like it on the condition that we decouple the stdlib versioning from the language definition versioning (even though I recommend we only officially support very specific combinations of the two). My suggestion is really just a concrete proposal for implementing Ezio's idea of only bumping the Python version for releases with long term support, and using some other mechanism to distinguish the interim releases.

IOW we would have a language moratorium every 2 years (i.e. between LTS releases) while switching to a 6 month release cycle for language/VM bugfixes and full stdlib releases? I would support that as it has several benefits from several angles.

From a VM perspective, it gives other VMs 2 years to catch up to the next release instead of 18 months; not a big switch, but still better than shortening it.

It also makes disruptive language changes less frequent so people have more time to catch up, update books/docs, etc. We can also let them bake longer and we all get more experience with them.

Doing a release every 6 months that includes updates to the stdlib and bugfixes to the language/VM also benefits other VMs by getting compatibility fixes in faster. All of the other VM maintainers have told me that keeping the stdlib non-CPython compliant is the biggest hurdle. This kind of switch means they could release a VM that supports a release 6 months or a year after a language change release (e.g. 1 to 2 releases in) so as to get changes in faster and lower the need to keep their own fork.

It should also increase the chances of external developers of projects being willing to become core developers and contributing their project to Python. If they get to keep a 6 month release cycle we could consider pulling in project like httplib2 and others that have resisted inclusion in the stdlib because painfully long (for them) wait between releases.

So, assuming a 2 year LTS cycle, the released versions up to February 2015 with my suggestion would end up being: >From the default branch: Python 3.3.0 + stdlib 12.08.0 (~August 2012) Python 3.4.0a1 + stdlib 14.08.0a1 (~February 2013) Python 3.4.0a2 + stdlib 14.08.0a2 (~August 2013) Python 3.4.0a3 + stdlib 14.08.0a3 (~February 2014) Python 3.4.0a4 + stdlib 14.08.0a4 (~2014) Python 3.4.0b1 + stdlib 14.08.0b1 (~2014) Python 3.4.0b2 + stdlib 14.08.0b2 (~2014) Python 3.4.0c1 + stdlib 14.08.0c1 (~2014) Python 3.4.0 + stdlib 14.08 (~August 2014) Python 3.5.0a1 + stdlib 16.08.0a1 (~February 2015) >From the 3.3 maintenance branch (these are maintenance updates to the "LTS" release): Python 3.3.1 + stdlib 12.08.1 (~February 2013) Python 3.3.2 + stdlib 12.08.2 (~August 2013) Python 3.3.3 + stdlib 12.08.3 (~February 2014) Python 3.3.4 + stdlib 12.08.4 (~August 2014) (and 3.3 branch enters security patch only mode) >From the 3.4 maintenance branch (these are maintenance updates to the "LTS" release): Python 3.4.1 + stdlib 14.08.1 (~February 2015) >From the stdlib feature development branch (these are the new interim releases with standard library updates only as proposed by PEP 407): Python 3.3.1 + stdlib 13.02.0 (~February 2013) Python 3.3.2 + stdlib 13.08.0 (~August 2013) Python 3.3.3 + stdlib 14.02.0 (~February 2014) (only upgrade path from here is to make the jump to 3.4.0) -- 3.4.0 + 12.08.0 is released from default branch -- Python 3.4.1 + stdlib 15.02.0 (~February 2015) If we have to make "brown paper bag" releases for the maintenance or stdlib branches then the micro versions get bumped - the date based version of the standard library versions relates to when that particular API was realised, not when bugs were last fixed in it. If a target release date slips, then the stdlib version would be increased accordingly (cf. Ubuntu 6.06). Yes, we'd have an extra set of active buildbots to handle the stdlib branch, but a) that's no harder than creating the buildbots for a new maintenance branch and b) the interim release proposal will need to separate language level changes from stdlib level changes anyway. As far as how sys.version checks would be updated, I would propose a simple API addition to track the new date-based standard lib versioning: sys.stdlibversion. People could choose to just depend on a specific Python version (implicitly depending on the stdlib version that was originally shipped with that version of CPython), or they may instead decide to depend on a specific stdlib version (implicitly depending on the first Python version that was shipped with that stdlib). The reason I like this scheme is that it allows us (and users) to precisely track the things that can vary at the two different rates. At least the following would still be governed by changes in the first two fields of sys.version (i.e. the major Python version): - deprecation policy - language syntax - compiler AST - C ABI stability - Windows compilation suite and C runtime version - anything else we decide to link with the Python language version (e.g. default pickle protocol) However, the addition of date based stdlib versioning would allow us to clearly identify the new interim releases proposed by PEP 407 without mucking up all those things that are currently linked to sys.version and really shouldn't be getting updated every 6 months. Users get a clear guarantee that if they follow the stdlib updates instead of the regular maintenance releases, they'll get nice new features along with their bug fixes, but no new deprecations or backwards incompatible API changes. However, they're also going to be obliged to transition to each new language release as it comes out if they want to continue getting security updates. Basically, what it boils down to is that I'm now +1 on the general proposal in the PEP, so long as: 1. We get a separate Hg branch for "stdlib only" changes and default becomes the destination specifically for "language update" changes (with the latter being a superset of the former) 2. The proposed "interim releases" are denoted by a new date-based sys.stdlibversion field and sys.version retains its current meaning (and slow rate of change) I don't think we need to do a new versioning scheme. Why can't we just say which releases are covered by a language moratorium? The community seemed to pick up on that rather well when we did it for Python 3 and I didn't see anyone having difficulty explaining it when someone didn't know what was going on. As long as we are clear which releases are under a language moratorium and which one's aren't we shouldn't need to switch to language + stdlib versioning scheme. This will lead to use reaching Python 4 faster (in about 4 years), but even that doesn't need to be a big deal. Linux jumped from 2 to 3 w/o issue. Once again, as long as we are clear on which new versions have language changes it should be clear as to what to expect.

Otherwise I say we just bump the major version when we do a language-changing release (i.e. every 2 years) and just to a minor/feature number bump (i.e. every 6 months) when we add/change stuff to the stdlib. People can then be told "learn Python 4" which is easy to point out on docs, e.g. you won't have to go digging for what minor/feature release a book covers, just what major release which will probably be emblazoned on the cover. And with the faster stdlib release schedule other VMs can aim for X.N versions when they have all the language features and all of their compatibility fixes into the stdlib. And then once they hit that they can just continue to support that major version by just keeping up with minor releases with compatibility fixes (which buildbots can help guarantee).

And honestly, if we don't go with this I'm with Georg's comment in another email of beginning to consider stripping the stdlib down to core libraries to help stop with the bitrot (sorry, Paul). If we can't attract new replacements for modules we can't ditch because of backwards compatibility I start to wonder if I should even care about improving the stdlib outside of core code required to make Python simply function.

-Brett

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20120118/54700086/attachment.html>

More information about the Python-Dev mailing list