[Python-Dev] 2.7 is here until 2020, please don't call it a waste. (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sat May 30 10:55:20 CEST 2015

On 30 May 2015 10:46, "Alexander Walters" <tritium-list at sdamon.com> wrote:

Python is a giant cache-miss generator. A little performance boost on the opt-code dispatch isn't going to change that much. If we really do care about improving python to do less environmental damage, then that is a discussion we should be having on it's own merits. It was really out of place, even in this tangenty thread.

I think the way core development gets funded is entirely on topic for the main core development mailing list, we just historically haven't discussed it openly, even though some of us have been advocating for various improvements to arrangements behind the scenes. I personally consider becoming more transparent about how we go about that process to be a good thing.

Intel are looking to get involved in CPython core development specifically to work on performance improvements, so it's important to offer folks in the community good reasons for why we're OK with seeing at least some of that work applied to Python 2, rather than restricting their contributions to Python 3.

The key is that the reason for not backporting performance enhancements differs from the reasons for not backporting new features. Rolling out new features has a broad ripple effect on the Python ecosystem as books, training material, etc, all need to be updated, and projects need to decide how to communicate their version dependencies appropriately if they decide to depend on one of the backported features. We pushed that kind of Python 2 change out to PyPI years ago, and aside from truly exceptional cases like the network security enhancements in PEPs 466 & 476 and the decision to bundle pip to make it easier to access PyPI, it isn't open for reconsideration as a general principle.

Performance improvements, by contrast, historically haven't been backported solely due to the stability and maintainability implications for CPython itself - they don't have a change management ripple effect the way new language and standard library level features do. That lack of negative ripple effects that cause work for other people is why the proposal to contribute paid development time makes such a big difference to the acceptability of Python 2.7 performance patches, as it should be a pure gain for current Python 2.7 users, and the paid development contributions should address the maintainability concerns on the core development side (particularly since Intel are paying for their coaching in core contribution practices and politics, rather than expecting to receive that coaching from community volunteers for free).

Backporting the computed goto patch is an easy place for them to start, since the change is already well tested in the Python 3 branch, but we don't expect this to be the end of the line for CPython 2 (or 3) performance enhancements.

However, we also shouldn't downplay the significance of this as a notable policy change for the Python 2.7 maintenance branch, which means it is useful to offer folks as many reasons as we can to help them come to terms with the idea that Python 2 performance still matters, and that it is only the limitations on our development and support capacity that prevented us from further improving it previously.

The commercially pragmatic reason is because Python 2 is where the largest current installed base is today, so applying some of the increased development capacity arising from sponsored contributions to Python 2.7 performance improvements is a good way to demonstrate to Python 2 developers that we still care about them as Python 2 users, rather than only being interested in them as potential future Python 3 users. This is the rationale that's likely to get our paid contributors (both current and future) on board with the idea, but it isn't necessarily going to be compelling to folks that are here as volunteers.

The first "What's in it for the volunteers?" reason is the one I raised: giving the nod to an increased corporate developer presence in Python 2 maintenance should eventually let volunteers stop worrying about even Python 2.7 bug fix changes with a clear conscience, confident that as volunteer efforts drop away redistributors and other folks with an institutional interest will pick up the slack with paid development time. "Do the fun stuff for free, figure out a way to get paid for the boring-but-necessary stuff (or leave those tasks to someone else that's getting paid to handle them)" is a good sustainable approach to open source development, while trying to do it all for free is a fast path to burnout.

Being ready, willing and able to handle the kind of situation created by the Python 2->3 community transition is a large part of what it means to offer commercial support for community driven open source projects, as it buys customers' time for either migration technologies to mature to a point where the cost of migration drops dramatically, for the newer version of a platform to move far enough ahead of the legacy version for there to be a clear and compelling business case for forward porting existing software, or (as is the case we're aiming to engineer for Python), both.

The environmental argument is another one that may be appealing to folks that have no commercial interest in improving Python 2 performance. Regardless of which programming language we use to write our own software, we all still share the same planet, so reducing the amount of power we collectively use is something we can all benefit from. Even though none of us have the necessary data to even guess at the absolute magnitude of that reduction, we can at least be confident it's a non-trivial portion of the amount of power Python 2 applications are currently consuming.

Regards, Nick.

More information about the Python-Dev mailing list