[Python-ideas] PEP 4XX: Adding sys.implementation (original) (raw)

Eric Snow ericsnowcurrently at gmail.com
Fri Apr 27 08:36:26 CEST 2012

I've written up a PEP for the sys.implementation idea. Feedback is welcome!

You'll notice some gaps which I'll be working on to fill in over the next couple days. Don't mind the gaps. They are in less critical (?) portions and I wanted to get this out to you before the weekend. Thanks!

-eric

PEP: 4XX Title: Adding sys.implementation Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Eric Snow <ericsnowcurrently at gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 26-April-2012 Python-Version: 3.3

Abstract

This PEP introduces a new variable for the sys module: sys.implementation. The variable holds consolidated information about the implementation of the running interpreter. Thus sys.implementation is the source to which the standard library may look for implementation-specific information.

The proposal in this PEP is in line with a broader emphasis on making Python friendlier to alternate implementations. It describes the new variable and the constraints on what that variable contains. The PEP also explains some immediate use cases for sys.implementation.

Motivation

For a number of years now, the distinction between Python-the-language and CPython (the reference implementation) has been growing. Most of this change is due to the emergence of Jython, IronPython, and PyPy as viable alternate implementations of Python.

Consider, however, the nearly two decades of CPython-centric Python (i.e. most of its existance). That focus had understandably contributed to quite a few CPython-specific artifacts both in the standard library and exposed in the interpreter. Though the core developers have made an effort in recent years to address this, quite a few of the artifacts remain.

Part of the solution is presented in this PEP: a single namespace on which to consolidate implementation specifics. This will help focus efforts to differentiate the implementation specifics from the language. Additionally, it will foster a multiple-implementation mindset.

Proposal

We will add sys.implementation, in the sys module, as a namespace to contain implementation-specific information.

The contents of this namespace will remain fixed during interpreter execution and through the course of an implementation version. This ensures behaviors don't change between versions which depend on variables in sys.implementation.

sys.implementation is a dictionary, as opposed to any form of "named" tuple (a la sys.version_info). This is partly because it doesn't have meaning as a sequence, and partly because it's a potentially more variable data structure.

The namespace will contain at least the variables described in the Required Variables_ section below. However, implementations are free to add other implementation information there. Some possible extra variables are described in the Other Possible Variables_ section.

This proposal takes a conservative approach in requiring only two variables. As more become appropriate, they may be added with discretion.

Required Variables

These are variables in sys.implementation on which the standard library would rely, meaning they would need to be defined:

name the name of the implementation (case sensitive).

version the version of the implementation, as opposed to the version of the language it implements. This would use a standard format, similar to sys.version_info (see Version Format_).

Other Possible Variables

These variables could be useful, but don't necessarily have a clear use case presently:

cache_tag a string used for the PEP 3147 cache tag (e.g. 'cpython33' for CPython 3.3). The name and version from above could be used to compose this, though an implementation may want something else. However, module caching is not a requirement of implementations, nor is the use of cache tags.

repository the implementation's repository URL.

repository_revision the revision identifier for the implementation.

build_toolchain identifies the tools used to build the interpreter.

url (or website) the URL of the implementation's site.

site_prefix the preferred site prefix for this implementation.

runtime the run-time environment in which the interpreter is running.

gc_type the type of garbage collection used.

Version Format

XXX same as sys.version_info?

Rationale

The status quo for implementation-specific information gives us that information in a more fragile, harder to maintain way. It's spread out over different modules or inferred from other information, as we see with platform.python_implementation().

This PEP is the main alternative to that approach. It consolidates the implementation-specific information into a single namespace and makes explicit that which was implicit.

With the single-namespace-under-sys so straightforward, no alternatives have been considered for this PEP.

Discussion

The topic of sys.implementation came up on the python-ideas list in 2009, where the reception was broadly positive [1]. I revived the discussion recently while working on a pure-python imp.get_tag() [2]. The messages in issue 14673_ are also relevant.

Use-cases

platform.python_implementation()

"explicit is better than implicit"

The platform module guesses the python implementation by looking for clues in a couple different sys variables [3]_. However, this approach is fragile. Beyond that, it's limited to those implementations that core developers have blessed by special-casing them in the platform module.

With ``sys.implementation` the various implementations would explicitly set the values in their own version of the sys module.

Aside from the guessing, another concern is that the platform module is part of the stdlib, which ideally would minimize implementation details such as would be moved to sys.implementation.

Any overlap between sys.implementation and the platform module would simply defer to sys.implementation (with the same interface in platform wrapping it).

Cache Tag Generation in Frozen Importlib

PEP 3147 defined the use of a module cache and cache tags for file names. The importlib bootstrap code, frozen into the Python binary as of 3.3, uses the cache tags during the import process. Part of the project to bootstrap importlib has been to clean out of Lib/import.c any code that did not need to be there.

The cache tag defined in Lib/import.c was hard-coded to "cpython" MAJOR MINOR [4]_. For importlib the options are either hard-coding it in the same way, or guessing the implementation in the same way as does platform.python_implementation().

As long as the hard-coded tag is limited to CPython-specific code, it's livable. However, inasmuch as other Python implementations use the importlib code to work with the module cache, a hard-coded tag would become a problem..

Directly using the platform module in this case is a non-starter. Any module used in the importlib bootstrap must be built-in or frozen, neither of which apply to the platform module. This is the point that led to the recent interest in sys.implementation.

Regardless of how the implementation name is gotten, the version to use for the cache tag is more likely to be the implementation version rather than the language version. That implementation version is not readily identified anywhere in the standard library.

Implementation-Specific Tests

XXX

http://hg.python.org/cpython/file/2f563908ebc5/Lib/test/support.py#l509 http://hg.python.org/cpython/file/2f563908ebc5/Lib/test/support.py#l1246 http://hg.python.org/cpython/file/2f563908ebc5/Lib/test/support.py#l1252 http://hg.python.org/cpython/file/2f563908ebc5/Lib/test/support.py#l1275

Jython's os.name Hack

XXX

http://hg.python.org/cpython/file/2f563908ebc5/Lib/test/support.py#l512

Impact on CPython

XXX

Feedback From Other Python Implementators

IronPython

XXX

Jython

XXX

PyPy

XXX

Past Efforts

XXX PEP 3139 XXX PEP 399

Open Issues

Implementation

The implementatation of this PEP is covered in issue 14673_.

References

.. [1]

http://mail.python.org/pipermail/python-dev/2009-October/092893.html

.. [2]

http://mail.python.org/pipermail/python-ideas/2012-April/014878.html

.. [3]

http://hg.python.org/cpython/file/2f563908ebc5/Lib/platform.py#l1247

.. [4]

http://hg.python.org/cpython/file/2f563908ebc5/Python/import.c#l121

.. _issue 14673

http://bugs.python.org/issue14673

Copyright

This document has been placed in the public domain.

Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

More information about the Python-ideas mailing list