[Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale (original) (raw)
INADA Naoki songofacandy at gmail.com
Sun May 7 01:22:51 EDT 2017
- Previous message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Next message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi, Nick.
After thinking about relationship between PEP 538 and 540 in two days, I came up with idea which removes locale coercion by default from PEP 538, it does just enables UTF-8 mode and show warning about C locale.
Of course, this idea is based on PEP 540. There are no "If PEP 540 is rejected".
How do you think?
If it make sense, I want to postpone PEP 538 until PEP 540 is accepted or rejected, or merge PEP 538 into PEP 540.
Background
Locale coercion in current PEP 538 has some downsides:
If user set
LANG=C LC_DATE=ja_JP.UTF-8, locale coercion may overrides LC_DATE.It makes behavior divergence between standalone and embedded Python.
Parent Python process may use utf-8:surrogateescape, but child process Python may use utf-8:strict. (Python 3.6 uses ascii:surrogateescape in both of parent and children).
On the other hand, benefits from locale coercion is restricted:
- When locale coercion succeeds, warning is always shown. To hide the warning, user must disable coercion in some way. (e.g. use UTF-8 locale explicitly, or set PYTHONCOERCECLOCALE=0).
So I feel benefit / complexity ratio of locale coercion is less than UTF-8 mode.
But locale coercion works nice on Android. And there are some Android-like Unix systems (container or small device) that C.UTF-8 is always proper locale.
Rough spec
Make Android-style locale coercion (forced, no warning) is now build option. Some users who build Python for container or small device may like it.
Normal Python build doesn't change locale. When python executable is run in C locale, show locale warning. locale warning can be disabled as current PEP 538.
User can disable automatic UTF-8 mode by setting PYTHONUTF8=0 environment variables. User can hide warning by setting PYTHONUTF8=1 too.
On Fri, May 5, 2017 at 10:21 PM, INADA Naoki <songofacandy at gmail.com> wrote:
On Fri, May 5, 2017 at 1:25 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
On Thu, 4 May 2017 11:24:27 +0900 INADA Naoki <songofacandy at gmail.com> wrote:
Hi, Nick and all core devs who are interested in this PEP.
I'm reviewing PEP 538 and I want to accept it in this month. It will reduces much UnicodeError pains which server-side OPs facing. Thank you Nick for working on this PEP. If you have something worrying about this PEP, please post a comment soon. If you don't have enough time to read entire this PEP, feel free to ask a question about you're worrying. From my POV, it is problematic that the behaviour outlined in PEP 538 (see Abstract section) varies depending on the adoption of another PEP (PEP 540). If we want to adopt PEP 538 before pronouncing on PEP 540, then PEP 538 should remove all points conditional on PEP 540 adoption, and PEP 540 should later be changed to adopt those removed points as PEP 540-specific changes. Regards Antoine. Fair enough. I stop hurrying about PEP 538 and start reviewing PEP 540. Thanks,
- Previous message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Next message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]