[Python-Dev] cpython: Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings: (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Apr 11 22:57:38 CEST 2013

2013/4/11 Serhiy Storchaka <storchaka at gmail.com>:

On 09.04.13 23:29, victor.stinner wrote:

http://hg.python.org/cpython/rev/53879d380313 changeset: 83216:53879d380313 parent: 83214:b7f2d28260b4 user: Victor Stinner <victor.stinner at gmail.com> date: Tue Apr 09 21:53:09 2013 +0200 summary: Add fast-path in PyUnicodeDecodeCharmap() for pure 8 bit encodings: cp037, cp500 and iso88591 codecs I deliberately specialized only most typical case in order to reduce maintaining cost. Further optimization of two not the most popular encodings probably not worth additional 25 lines of code.

I did the commit while I was trying to avoid usage of PyUnicode_READ_CHAR() and PyUnicode_READ() in unicodeobject.c (slow macros). I was surprised that PyUnicode_DecodeCharmap() has a fast path for Py_UCS2 mapping but not Py_UCS1 mapping.

After implementing the fast-path, I realized that only a very few codecs use it.

So what do you suggest? Revert the commit to restore the following "hack" (to only have one fast-path)?

'\ufffe' ## Widen to UCS2 for optimization

The Py_UCS1 fast-path has a small advantage: decoding cannot fail (no need to call an error handler).

Victor

More information about the Python-Dev mailing list