[Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Tue May 23 12:34:50 EDT 2017

On 23 May 2017 at 18:38, INADA Naoki <songofacandy at gmail.com> wrote:

Hi, Nick.

I read again and I think PEP 538 is mostly ready for accepted, without waiting PEP 540.

Great to hear!

One remaining my concern is setting LANG.

Setting LANG to C.UTF-8 ensures that even components that only check the LANG fallback for their locale settings will still use C.UTF-8 . https://www.python.org/dev/peps/pep-0538/#setting-both-lc-ctype-lang-for-utf-8-locale-coercion I feel setting only LCCTYPE making PEP 538 simpler. Is there any real component using LANG for deciding encoding? For example, date command refers LCTIME. $ LANG=jaJP.UTF-8 LCCTYPE=C date 2017年 5月 23日 火曜日 17:31:03 JST $ LANG=jaJP.UTF-8 LCCTYPE=C.UTF-8 date # Coercing only LCCTYPE 2017年 5月 23日 火曜日 17:32:58 JST $ LANG=C.UTF-8 LCCTYPE=C.UTF-8 date # Coercing both of LCCTYPE and LANG Tue May 23 17:31:10 JST 2017 In this case, coercing only LCCTYPE has less side-effect. Would you add example demonstrates how coercing LANG helps people?

I'm honestly not sure it does - I think it's an assumption I added to the PEP early on, and never actually tested. Looking at it more closely now, all of the interpreter level checks are specifically for LC_CTYPE, and experimenting with "LANG=C LC_CTYPE=C.UTF-8" indicates that coercing only LC_CTYPE is still enough to fix the GNU readline encoding compatibility problem covered in https://www.python.org/dev/peps/pep-0538/#considering-locale-coercion-independently-of-utf-8-mode

So I'll take another pass through the implementation this weekend, and simplify it to only set LC_CTYPE regardless of whether it's using C.UTF-8, C.utf8, or UTF-8 as the target locale. Assuming that doesn't uncover any hidden problems with the idea, I'll then update the PEP to match.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list