[Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale (original) (raw)

INADA Naoki songofacandy at gmail.com
Sun Mar 5 09:39:20 EST 2017

LGTM and I love this PEP and PEP 540.

Some comments:

...

* PEP 540 proposes to entirely decouple CPython's default text encoding from the C locale system in that case, allowing text handling inconsistencies to arise between CPython and other C/C++ components running in the same process and in subprocesses. This approach aims to make CPython behave less like a locale-aware C/C++ application, and more like C/C++ independent language runtimes like the JVM, .NET CLR, Go, Node.js, and Rust

I prefer just "locale-aware" / "locale-independent" (application | library | function) to "locale-aware C/C++ application" / "C/C++ independent" here.

Both of Rust and Node.JS are linked with libc. And Node.JS (v8) is written in C++. They just demonstrates many people prefer "always UTF-8" to "LC_CTYPE aware encoding" in real world application.

And C/C++ can be used for locale-aware and locale-independent application. I can print "こんにちは、世界" in C locale, because stdio is byte transparent. There are many locale independent libraries written in C (zlib, libjpeg, etc..), and some functions in libc are locale-independent or LC_CTYPE independent (printf is locale-aware, but it uses LC_NUMERIC, not LC_CTYPE).

...

Backporting to Python 3.6.0 ---------------------------

If this PEP is accepted for Python 3.7, redistributors backporting the change specifically to their initial Python 3.6.0 release will be both allowed and encouraged. However, such backports should only be undertaken either in conjunction with the changes needed to also provide a suitable locale by default, or else specifically for platforms where such a locale is already consistently available.

If it's really encouraged, how about providing patch officially, or backport it in 3.6.2 but disabled by default? Some Python users (including my company) uses pyenv or pythonz to build Python from source. This PEP and PEP 540 are important for them too.

More information about the Python-Dev mailing list