GitHub - elpython/elpython-meta: Meta-tracker for the ELPython project (original) (raw)

What is EL Python?

Right now, EL Python is just an idea in a README in a GitHub repo. Whether it will ever become more than that remains to be seen :)

What is the idea?

EL Python would be a CPython 3.x variant tailored for Enterprise Lifecycles. It would exist as a venue for paid developers working for commercial redistributors and other organisations to collaborate on:

By defining EL Python as a separate project from CPython, we'd be able to make it clear that we_don't_ consider it reasonable to ask volunteer community contributors to invest their personal time in meeting the needs of large enterprise organisations, but we do consider it reasonable for commercial Python redistributors and other organisations using Python to invest in providing a CPython variant that promises:

How would this idea be pursued?

If this idea ends up being pursued further, then this README would be redrafted in the form of a Python Enhancement Proposal and submitted to python-dev for consideration.

Actually taking that step would be conditional on credible commitments of full-time developer support from interested commercial Python redistributors and other organisations :)

How often would new EL Python releases happen?

EL Python releases would be made on a six-monthly cadence, with the releases deliberately timed to align with the development schedules of the Fedora and Ubuntu Linux distributions.

Given those dates, the target dates for EL Python releases would be the final week of January and the final week of July each year.

Which CPython branches would EL Python track?

By default, there would be an EL Python maintenance branch for each CPython maintenance branch. These branches would all include a core set of patches that:

Each branch would also include a set of approved backports from later CPython maintenance and feature releases.

In addition, at the EL Python maintainers' discretion, some EL Python maintenance branches may be defined as LTS branches, which would live on beyond the mainstream EOL of the corresponding CPython branches.

Who would have merge permissions for EL Python branches?

At least initially, EL Python merge permissions would be restricted to staff working for commercial redistributors and other organisations formally participating in the EL Python effort (where the exact nature of "formal participation" would be defined in collaboration with the Python Software Foundation).

This is due to the fact that the proposed differences in maintenance policies between EL Python and CPython are driven almost entirely by commercial and enterprise support requirements, which gives EL Python a strongly business-driven character that isn't the case for the more egalitarian, community-driven, CPython project.

It's expected that this restriction would eventually be dropped - it's just a simplfying community management assumption to start out with.

How would EL Python versions be identified?

It is proposed that EL Python versions be identified through a combination of semantic versioning and calendar versioning:

This information will be reported as:

To maximise backwards compatibility (and to ensure that otherwise CPython specific test cases are run), sys.implementation.name will be reported as "cpython" in EL Python branches, and the default CPython sys.implementation.cache_tag will also be re-used.

Even if no other markers were added, this would be sufficient to allow CPython and EL Python builds to be fairly reliably distinguished based on:

impl = sys.implementation
if impl.name == 'cpython':
    if impl.version == sys.version_info:
        print("CPython")
    else:
        print("EL Python")

However, as a more robust marker that also allows for identification of other CPython variants, an additional implementation-specific field, variant would be added:

This would be proposed as a new standard sys.implementation field for Python 3.9+ (defaulting to reference for reference implementations, such as CPython itself for the sys.implementation.name == 'cpython' case).

Redistributors would be encouraged to always set it appropriately to help identify their particular builds (using the elpython- prefix for EL Python rebuilds).

What does "no mandatory file regeneration" mean?

Migrating to a new CPython feature release necessarily means doing the following:

These updates are required even if the affected software is otherwise entirely compatible with both runtime versions at a source code level.

The "no mandatory rebuilds or file regeneration" requirement means that:

  1. Any EL Python branch will always support the filesystem layout of the corresponding CPython X.Y.Z release
  2. Any EL Python branch will always support loading pyc files and extension modules named with the compatibility tags for the corresponding CPython X.Y.Z release
  3. Any EL Python branch will always support installing wheel files using the compatibility tags for the corresponding CPython X.Y.Z release

Features and fixes will thus only be considered as potential backport candidates from CPython to EL Python if either:

How would EL Python be expected to affect the testing matrix for community Python projects?

Any software tested against a given CPython X.Y.Z release would be expected to work unmodified on the corresponding EL Python branch. Software tested against a later CPython X.Y.Z+ release would normally be expected to work against the corresponding EL Python release, as long as it was not impacted by any bugs that had been fixed in CPython, but the fix hadn't been backported due to backwards compatibility concerns that were deemed acceptable for CPython, but not for EL Python.

This means that for projects which already promptly drop support for older CPython versions when new feature releases become available, or when a given maintenance branch is no longer receiving security updates, EL Python should have no impact on the typical testing matrix (and this is considered a desirable outcome).

Where EL Python may potentially become an interesting testing target for Python community projects is in the following cases:

The goal of this approach is that the existence of EL Python should never increase the testing burden for any community project - if EL Python is added to a testing matrix, it should typically be accompanied by the removal of the corresponding CPython branch (either because that branch is no longer receiving security updates, or because it lacks a feature the project is now relying on).

By contrast, both previous proposals for making new CPython features available to end users without waiting for the next CPython feature release had the side effect of potentially significantly expanding build & test matrices for community projects (see the deferredPEP 407 and the withdrawnPEP 413 for details)

What is the proposed relationship with CPython?

All functional changes to EL Python should either appear in CPython first, or else be covered by a draft PEP or feature request that has been submitted for consideration for a future version of CPython, and python-dev has made the determination that it is suitable for inclusion as an EL Python feature, pending further consideration as a potential mainline CPython feature.

To ensure the absence of churn-related compatibility breaks in EL Python maintenance releases, EL Python may also end up including opt-in behaviour toggles that don't appear in regular CPython feature releases (see PEP 493 for some examples of this kind of toggle related to backporting the PEP 476 change to always verify HTTPS certificates by default).

To allow for transparent upgrades from a regular CPython build to EL Python, the directory paths used in EL Python would necessarily be fully compatible with those used by CPython (e.g. the /usr/liband /usr/lib subdirectories would still be called pythonX.Y). However, EL Python may also add additional EL Python specific directories that a regular CPython build will ignore.

How else would EL Python differ from CPython?

One big difference would be that EL Python would be a source-only project: it would not provide any prebuilt binary installers for any platform. Instead, the expectation would be that redistributors and end users would take the project and produce their own binaries in the formats of interest to them.

In addition, while EL Python would transparently consume pyc files, extension modules and wheel files built for CPython, and would default to emitting CPython compatible versions of such files, a mechanism would need to be provided for publishers to indicate when their artifacts were EL Python specific (e.g. when they're depending on a feature backported from a later CPython release).

What is the proposed relationship with the PSF?

We'd like EL Python to be a PSF backed project, just like CPython, and contributors to EL Python would be required to sign the CPython CLA before their contributions can be accepted.

We'd also expect any formal participation agreements to be worked out in collaboration with the PSF, rather than directly between participating organisations.

What Code of Conduct would the project use?

EL Python would operate under the Contributor Covenant: https://www.contributor-covenant.org/

How would the CoC be enforced?

Initially, by the folks with merge privileges, with their respective employers and the PSF Board available as potential paths of escalation for any CoC concerns relating to the project maintainers themselves. EL Python would also respect any existing org-wide bans from PSF provided communication channels.

Eventually, as the PSF defines clearer processes for reporting of CoC concerns for the projects it oversees, EL Python would explicitly opt in to being covered by that process.

What are the alternatives to pursuing the EL Python idea?

Historically, redistributors (most notably Red Hat, as part of Red Hat Enterprise Linux) have offered long term commercial support for CPython simply by choosing a particular CPython X.Y.Z release, and then offering that specific version for an extended period of time, focusing mainly on security fixes, backports of requested bug fixes, and occasionally backports of smaller features (e.g. backporting collections.OrderedDict from Python 2.7 to Python 2.6 in RHEL & CentOS 6).

An example of this approach can be seen by looking at the patch set for Python 2.7 in CentOS 7, which relies on selective backports atop 2.7.5 rather than rebasing outright to upstream maintenance releases: https://git.centos.org/rpms/python/blob/c7/f/SOURCES

Unfortunately, while this approach does address the key concern of providing a stable, long-lived foundation for enterprise developers to build on, many other aspects of the related developer experience are considered highly undesirable:

The EL Python concept arose from asking the question of how we might effectively address the established commercial demand for a long term support variant of CPython in a way that:

It is based in large part on the three tiered concept of "development", "stable", and "long term" kernels in the Linux kernel development process:

What about Python 2?

While EL Python itself is being proposed as a Python 3 only project, the tauthon project is already pursuing a similar concept for Python 2.7: https://github.com/naftaliharris/tauthon

If EL Python were to ever gain a Python 2.7 branch, it would only be in the form of a post-2020 security-fix branch (including network security stack upgrades), rather than the kind of feature backport branch being proposed for 3.x releases.