[Python-Dev] Support of UTF-16 and UTF-32 source encodings (original) (raw)

Paul Moore p.f.moore at gmail.com
Sun Nov 15 07:56:18 EST 2015

On 15 November 2015 at 07:23, Stephen J. Turnbull <stephen at xemacs.org> wrote:

I don't see any good reason for allowing non-ASCII-compatible encodings in the reference CPython interpreter.

From PEP 263:

   Any encoding which allows processing the first two lines in the
   way indicated above is allowed as source code encoding, this
   includes ASCII compatible encodings as well as certain
   multi-byte encodings such as Shift_JIS. It does not include
   encodings which use two or more bytes for all characters like
   e.g. UTF-16. The reason for this is to keep the encoding
   detection algorithm in the tokenizer simple.

So this pretty much confirms that double-byte encodings are not valid for Python source files.

Paul

More information about the Python-Dev mailing list