[Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale (original) (raw)
Nick Coghlan ncoghlan at gmail.com
Tue Mar 14 22:29:17 EDT 2017
- Previous message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Next message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On 15 March 2017 at 06:22, Chris Barker <chris.barker at noaa.gov> wrote:
So the question nis -- is anyone counting on errors in this case? i.e., is a sysadmin thinking:
"I want an ASCII-only system, so I'll set the locale, and now I can expect any program running on this system that is not ascii compatible to fail." I honestly don't know if this is common -- but I would argue that trying to run a unicode-aware program on an ASCII-only system could be considered a mis-configuration as well.
From a mainstream Linux point of view, it's not common - on systemd-managed systems, for example, the only way to get the C locale these days is to either specify it in /etc/locale.conf, or to set it specifically in the environment. Upstart was a little less reliable about that, and sysvinit was less reliable still, but the trend is definitely towards making C.UTF-8 the assumed default, rather than "C". Even glibc itself would quite like to get to a point where you only get the C locale if you explicitly ask for it: https://sourceware.org/glibc/wiki/Proposals/C.UTF-8
The main practical objection that comes up in relation to "UTF-8 everywhere" isn't to do with UTF-8 per se, but rather with the size of the collation tables needed to do "proper" sorting of Unicode code points. However, there's a neat hack in the design of UTF-8 where sorting the encoded bytes by byte value is equivalent to sorting the decoded text by the Unicode code point values, which means that "LC_COLLATE=C" sorting by byte value, and "LC_COLLATE=C.UTF-8" sorting by "Unicode code point value" give the same results.
Cheers, Nick.
-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20170315/8d869f86/attachment.html>
- Previous message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Next message (by thread): [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]