[Python-Dev] PyPy 1.7 - widening the sweet spot (original) (raw)

Terry Reedy tjreedy at udel.edu
Tue Nov 22 23:20:40 CET 2011

On 11/22/2011 10:35 AM, Stefan Behnel wrote:

Maciej Fijalkowski, 22.11.2011 15:46:

2011/11/21 Terry Reedy:

I strongly recommend that where it makes a difference, the pypy python3 project target 3.3. In particular, don't reproduce the buggy narrow-build behavior of 3.2 and before (perhaps pypy avoids this already). Do include the new unicode capi in cpyext. I anticipate that 3.3 will see more production use than 3.2 [snip}

PyPy's py3k branch targets Python 3.2 until 3.3 is released and very likely 3.3 afterwards. Optimizations are irrelevant really in the case of PyPy.

I admit that I wasn't very clear in my wording. As Terry pointed out, the Unicode changes in Py3.3 are not only for speed and/or memory performance improvements, they also improve the compliance of the Unicode implementation, which implies a behavioral change.

One of the major features of Python 3 is the expansion of the directly supported character set from ascii to unicode. Python's original narrow and wide build unicode implementation has problems that were somewhat tolerable in an optional, alternate text class but which are much less so for the text class. The general problem is that the two builds give different answers for operations and functions on strings containing non-BMP characters. This differences potentially affects anything that uses strings, such as the re module, without guarding against the differences.

One can view the narrow build results as wrong and buggy. Extended chars were practically non-existent when the implementation was written, but are becoming more common now and in the future. In any case, Python string code no longer works the same across all x.y builds. On *nix platforms that can have both narrow and wide builds, there can also be binary conflicts for extension modules. On Windows, there is no conflict because one is stuck with a buggy narrow build. This is all besides the space issue.

In my view, Python 3.3 will be the first fully satisfactory Python 3 version. It should be the version of choice for any app doing full unicode text or document processing on platforms that include, in particular, Windows.

Since PyPy appears to have implemented the current wide/narrow behavior of Py2 and Py3.[012] already, I see no reason not to target 3.2 for the time being as it does not require substantial changes in this part. Compliance with Py3.3 will then require implementing the new behavior.

Thinking about how PyPy will do that should start well before 3.3 is released. My impression from reading the PyPy and Python 3 page, linked in the original post, is that releasing PyPy fully ready for Python 3, with all listed phases completed, will take close to a year anyway. Hence my comment.

-- Terry Jan Reedy

More information about the Python-Dev mailing list