Issue 20087: Mismatch between glibc and X11 locale.alias (original) (raw)
Created on 2013-12-28 09:29 by serhiy.storchaka, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (33)
Author: Serhiy Storchaka (serhiy.storchaka) * 
Date: 2013-12-28 09:29
The locale module uses locale alias table derived from X11 locale.alias file for mapping bare locale names without encodings to locale names with encodings. However sometimes glibc default encoding for a locale differs from that used in X11 locale.alias.
Here is full differences table:
GLibc X11 locale.aliasaz_az az_AZ.UTF-8 az_AZ.ISO8859-9E ca_ad ca_AD.ISO8859-15 ca_AD.ISO8859-1 ca_fr ca_FR.ISO8859-15 ca_FR.ISO8859-1 ca_it ca_IT.ISO8859-15 ca_IT.ISO8859-1 cy_gb cy_GB.ISO8859-14 cy_GB.ISO8859-1 en_in en_IN.UTF-8 en_IN.ISO8859-1 et_ee et_EE.ISO8859-1 et_EE.ISO8859-15 fi_fi fi_FI.ISO8859-1 fi_FI.ISO8859-15 gd_gb gd_GB.ISO8859-15 gd_GB.ISO8859-1 hi_in hi_IN.UTF-8 hi_IN.ISCII-DEV iu_ca iu_CA.UTF-8 iu_CA.NUNACOM-8 iw_il iw_IL.ISO8859-8 he_IL.ISO8859-8 ka_ge ka_GE.GEORGIAN_PS ka_GE.GEORGIAN-ACADEMY lo_la lo_LA.UTF-8 lo_LA.MULELAO-1 mi_nz mi_NZ.ISO8859-13 mi_NZ.ISO8859-1 nr_za nr_ZA.UTF-8 nr_ZA.ISO8859-1 nso_za nso_ZA.UTF-8 nso_ZA.ISO8859-15 ru_ru ru_RU.ISO8859-5 ru_RU.UTF-8 rw_rw rw_RW.UTF-8 rw_RW.ISO8859-1 sq_al sq_AL.ISO8859-1 sq_AL.ISO8859-2 ss_za ss_ZA.UTF-8 ss_ZA.ISO8859-1 ta_in ta_IN.UTF-8 ta_IN.TSCII-0 tg_tj tg_TJ.KOI8_T tg_TJ.KOI8-C th_th th_TH.TIS_620 th_TH.ISO8859-11 tn_za tn_ZA.UTF-8 tn_ZA.ISO8859-15 ts_za ts_ZA.UTF-8 ts_ZA.ISO8859-1 tt_ru tt_RU.UTF-8 tt_RU.TATAR-CYR ur_pk ur_PK.UTF-8 ur_PK.CP1256 uz_uz uz_UZ.ISO8859-1 uz_UZ.UTF-8 uz_uz@cyrillic uz_UZ.UTF-8@cyrillic uz_UZ.UTF-8 vi_vn vi_VN.UTF-8 vi_VN.TCVN zh_cn zh_CN.GB2312 zh_CN.gb2312 zh_tw zh_TW.BIG5 zh_TW.big5 zh_tw.euctw zh_TW.EUC_TW zh_TW.eucTW
For example with the en_IN encoding:
import locale, _locale _locale.setlocale(locale.LC_CTYPE) 'en_IN' locale.getlocale() ('en_IN', 'ISO8859-1') locale.nl_langinfo(locale.CODESET) 'UTF-8' locale.setlocale(locale.LC_CTYPE, locale.getlocale()) Traceback (most recent call last): File "", line 1, in File "/home/serhiy/py/cpython/Lib/locale.py", line 592, in setlocale return _setlocale(category, locale) locale.Error: unsupported locale setting
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-03 08:06
Needed a test for few common locales (en_IN, ru_RU) and maybe for unusual locales (uz_uz, uz_uz@cyrillic).
I would prefer to have a separate issue that updates the aliases table to glibc 2.24.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-07 16:53
I agree that it's reasonable to have glibc's aliases override the X.org ones, but this patch makes some pretty significant changes to Python's default assumptions with respect to default encodings for several locales.
While some changes obviously make sense (e.g. 'ca_AD.ISO8859-1' to 'ca_AD.ISO8859-15'), others are less clear (e.g. 'cy_GB.ISO8859-1' to 'cy_GB.ISO8859-14' or 'tg_TJ.KOI8-C' to 'tg_TJ.KOI8-T' or several of the moves from ISO encodings to UTF-8). Is there some reference for why glibc chose different values than X.org for these ?
I also don't understand why some "xx.utf-8" locale mappings were removed - I don't think we should remove those, unless they are no lot needed due to some other logic implying these mappings.
Since these are major changes, we need an appropriate warning in the NEWS file (and the "What's New" document), an update of the top comment (under "### Database") to mention that the glibc database takes precedence and where to find it,
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-07 17:23
'cy_GB.ISO8859-1' to 'cy_GB.ISO8859-14'
Looks as just fixing an error. The default West-European ISO8859-1 is changed to Celtic cy_GB.ISO8859-14. This looks better option for Welsh.
'tg_TJ.KOI8-C' to 'tg_TJ.KOI8-T'
KOI8-C is not supported by Python, but KOI8-T is supported. I don't know what KOI8-C means, there are several rarely used incompatible encodings with this name.
I also don't understand why some "xx.utf-8" locale mappings were removed - I don't think we should remove those, unless they are no lot needed due to some other logic implying these mappings.
The aliases table is a table of exceptions. Removed entries no longer are exceptional.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-07 18:29
On 07.03.2017 18:23, Serhiy Storchaka wrote:
Serhiy Storchaka added the comment:
'cy_GB.ISO8859-1' to 'cy_GB.ISO8859-14'
Looks as just fixing an error. The default West-European ISO8859-1 is changed to Celtic cy_GB.ISO8859-14. This looks better option for Welsh.
'tg_TJ.KOI8-C' to 'tg_TJ.KOI8-T'
KOI8-C is not supported by Python, but KOI8-T is supported. I don't know what KOI8-C means, there are several rarely used incompatible encodings with this name.
While all this may make sense, I'm missing some more reasoning behind the differences between X.org and glibc.
This change also looks strange:
- 'ka_ge': 'ka_GE.GEORGIAN-ACADEMY',
- 'ka_ge': 'ka_GE.GEORGIAN_PS', 'ka_ge.georgianacademy': 'ka_GE.GEORGIAN-ACADEMY', 'ka_ge.georgianps': 'ka_GE.GEORGIAN-PS', 'ka_ge.georgianrs': 'ka_GE.GEORGIAN-ACADEMY',
Why is GEORGIAN_PS written with an underscore whereas the other mappings use dashes ?
Or this one:
- 'fi_fi': 'fi_FI.ISO8859-15',
- 'fi_fi': 'fi_FI.ISO8859-1',
Why would a locale switch away from an encoding having the Euro sign to one without it ?
Or why is this latin variant removed:
Why should Russians switch back to ISO ?
- 'ru_ru': 'ru_RU.UTF-8',
- 'ru_ru': 'ru_RU.ISO8859-5',
or from ISO to KOI ?
- 'russian': 'ru_RU.ISO8859-5',
- 'russian': 'ru_RU.KOI8-R',
The more I look at these changes, the more I believe we should not simply take everything we find in the files for granted. They obviously both have bugs.
I also don't understand why some "xx.utf-8" locale mappings were removed - I don't think we should remove those, unless they are no longer needed due to some other logic implying these mappings.
The aliases table is a table of exceptions. Removed entries no longer are exceptional.
It's not a table of exceptions, it's a table mapping commonly used locale settings to ones which the lib C understands :-)
But regardless, I checked the code and it is already smart enough to convert lib C incompatible spellings such as "utf8" to "UTF-8", so these entries can indeed be removed, but only if the locale is otherwise listed.
In some cases, it's probably better to drop the ".utf8" to have more generic mappings, e.g.
- 'bhb_in.utf8': 'bhb_IN.UTF-8',
or
'de_li.utf8': 'de_LI.UTF-8',though I'd expect that mapping to be:
'de_li': 'de_LI.ISO8859-1',as for all other "de" entries.
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2017-03-08 06:27
Why is the X11 locale alias map used at all? It seems like it can only create confusion with libc.
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-08 07:20
Not all platforms use glibc 2.24 as libc.
Ideally most of entries should even not exist. We should ask libc for the default encoding if it is not included in the locale name. The aliases table should be used only for mapping commonly used but unsupported by libc locales to supported by libc locales.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-08 09:25
On 08.03.2017 08:20, Serhiy Storchaka wrote:
Serhiy Storchaka added the comment:
Not all platforms use glibc 2.24 as libc.
True. Many don't even use glibc.
Ideally most of entries should even not exist. We should ask libc for the default encoding if it is not included in the locale name. The aliases table should be used only for mapping commonly used but unsupported by libc locales to supported by libc locales.
I think you have a wrong understanding of what this alias table is used for: we need it to determine the lib C compatible locale name without using lib C APIs such as setlocale(), since these are not thread safe and have side-effects for the whole process.
The alias table is there to avoid having to go to the lib C to ask it indirectly for more details. Unfortunately, there are no cross-platform lib C APIs which would allow querying these details without also changing the local settings of the process.
I know that Python still plays the usual "save current locale, run setlocale(), revert to previous locale" trick in a couple of places and this works if Python is the only thread running, but it doesn't when embedded into other applications.
Regarding the patch: we cannot simply use the output from the script to set new values. The changes have to be manually reviewed as well.
E.g. this entry in the table is clearly a typo:
'en_zw.utf8': 'en_ZS.UTF-8',(it should read en_ZW.UTF-8)
This entry appears wrong as well:
'eo': 'eo_XX.ISO8859-3',(XX is not a valid country ISO code)
How should we go about this ? Mark all the problems in the PR ?
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-08 09:37
The problem is that that table can get incorrect result for non-Linux platforms (or for Linux with old glibc).
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-08 11:35
On 08.03.2017 07:27, Benjamin Peterson wrote:
Why is the X11 locale alias map used at all? It seems like it can only create confusion with libc.
Because it was the only such maintained mapping available at the time. It's also used for the X.org system, which has a rather strong focus on user interfaces where locale matter a lot, unlike the lib C :-)
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-08 11:48
On 08.03.2017 10:37, Serhiy Storchaka wrote:
The problem is that that table can get incorrect result for non-Linux platforms (or for Linux with old glibc).
Sure, it's a best effort approach.
Also note that on today's systems you often don't have the full set of locales available anymore - instead these have to either be installed separately or generated on the target system.
Our locale database works on all these system, regardless of what's installed or not.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-08 15:33
Why was the PR merged while we were still discussing it ?
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2017-03-09 07:15
"eo_XX" is just something that appears in the X11 locale.alias file. My change doesn't add that; it was already there. (for Esperanto, which I suppose explains the "XX")
Most of the changes you identify the glibc aliases taking precedence over the X11 ones. e.g., glibc has "fi_FI ISO-8859-1" while the X11 locale list has "fi_FI.ISO8859-15". That seems correct to me as far as the intent of this change is concerned.
How do you propose to pick and choose what we use from the X11 locale alias list?
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-09 10:23
On 09.03.2017 08:15, Benjamin Peterson wrote:
"eo_XX" is just something that appears in the X11 locale.alias file. My change doesn't add that; it was already there. (for Esperanto, which I suppose explains the "XX")
Yes, I know. That was an example of a bug in the X.org list.
Most of the changes you identify the glibc aliases taking precedence over the X11 ones. e.g., glibc has "fi_FI ISO-8859-1" while the X11 locale list has "fi_FI.ISO8859-15". That seems correct to me as far as the intent of this change is concerned.
No, it's not correct. ISO-8859-1 is the older version of Latin-1 without the Euro sign. ISO8859-15 adds it.
How do you propose to pick and choose what we use from the X11 locale alias list?
We have to go through the list one by one to check whether the mapping update makes sense and is correct.
This will be difficult in a few cases where the glibc mapping switches to UTF-8 from an ISO encoding. We'll have to find evidence that this change does indeed make sense.
My take on this is that the X.org folks know better than the glibc folks, since the former have to deal with end users that rely on the locale settings a lot more than applications using glibc for getting an initial locale setting right.
Also note that you are parsing the SUPPORTED file from glibc (in slightly processed form):
https://github.com/bminor/glibc/blob/master/localedata/SUPPORTED
This file does not provide a locale alias mapping as the routine in makelocalealias.py suggests. Instead it's a list of locales to install by default:
https://github.com/bminor/glibc/blob/73dfd088936b9237599e4ab737c7ae2ea7d710e1/localedata/Makefile
In glibc you can define both the locale and the encoding separately when creating a locale using localedef and the file simply provides the default parameters to pass to this tool.
As such, I don't see how you can derive a default alias meaning from the file.
It's simply an indication of what glibc would have installed in case it were installed from source, but that's hardly ever the case. On today's systems only a bare subset of locales is installed and more added as necessary, so you rarely have all the locales defined in SUPPORTED installed on a system.
So the file doesn't even provide a hint at what could be installed on the system ("locale -a" gives you that list).
Here's the history:
https://github.com/bminor/glibc/commits/master/localedata/SUPPORTED
It's merely a list of additions and removals from the default set. Nothing more. It does provide a list of known and supported locales, but no usable or authoritative encoding information (locales are defined using Unicode, so the encoding is a parameter and not predefined).
Overall, I believe the file is pretty useless to use as basis for an alias table providing encoding information. It may provide some ideas for corrections, but should not override the X.org one by default.
On the other hand, you have the local.alias master file:
https://cgit.freedesktop.org/xorg/lib/libX11/tree/nls/locale.alias.pre
together with the history of why changes were made and when. This is an authoritative resource and people are making changes against it from the user perspective.
I'd suggest to make the override optional in makelocalealias.py via a command line switch and to use this for manually adding or fixing X.org entries.
If you absolutely want to parse the glibc file per default as well, please only let it add new entries, not override existing ones. As we've seen in the patch, those overrides need to be carefully reviewed.
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-09 10:41
Why is the X11 locale alias map used at all? It seems like it can only create confusion with libc.
Originally only the X11 locale alias map was used. The support of the glibc locale alias map was added 2.5 years ago ().
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-09 10:47
The SUPPORTED file from glibc is used for determining the default encoding for locales that don't include it explicitly. For example en_IN uses UTF-8 rather than ISO8859-1.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-09 12:42
On 09.03.2017 11:47, Serhiy Storchaka wrote:
The SUPPORTED file from glibc is used for determining the default encoding for locales that don't include it explicitly. For example en_IN uses UTF-8 rather than ISO8859-1.
No, the glibc locales don't say anything about default encodings used in a locale:
http://manpages.ubuntu.com/manpages/wily/en/man5/locale.5.html
These encodings are just used for determining the default set of locale.encoding variants to install on the system, nothing more:
glibc does have a locale.alias file:
https://github.com/bminor/glibc/blob/73dfd088936b9237599e4ab737c7ae2ea7d710e1/intl/locale.alias
which uses the X.org format, but this is completely out of date and declared obsolete.
Serhiy: If you believe that there's anything authoritative about the glibc SUPPORTED file in terms of defining the commonly used encoding in a locale, please provide references. These should also clarify why the glibc encoding is the correct one compared to the X.org mapping.
It doesn't help, trying to interpret things into such build files. We need a database that is being actively maintained and has a track record of representing what people actually use in their locales. The only one I know is the X.org one.
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-09 13:07
The original issue is . The locale module returned encoding ISO8859-1 for locale en_IN (as in the X11 locale alias map), but glibc uses UTF-8 (as in glibc SUPPORT file).
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2017-03-10 07:37
Do you believe this program should work?
import locale, os
for l in open("/usr/share/i18n/SUPPORTED"):
alias, encoding = l.strip().split()
locale.setlocale(locale.LC_ALL, alias)
try:
enc = locale.getlocale()[1]
except ValueError:
continue # not in table
normalized = enc.replace("ISO", "ISO-").
replace("_", "-").
replace("euc", "EUC-").
replace("big5", "big5-").upper()
assert normalized == locale.nl_langinfo(locale.CODESET)
After my change it does—the encoding returned from getlocale() is the one actually being used by glibc. It fails dramatically on earlier versions of Python (for example on the en_IN example from #29571.) I don't understand why Python needs to editorialize whatever choices libc or the system administrator has made.
Is getlocale() expected to return something different from the underlying C locale?
In fact, why have this table at all instead of using nl_langinfo to return the encoding for the current locale?
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-10 15:29
On 10.03.2017 08:37, Benjamin Peterson wrote:
Do you believe this program should work?
import locale, os for l in open("/usr/share/i18n/SUPPORTED"): alias, encoding = l.strip().split() locale.setlocale(locale.LC_ALL, alias) try: enc = locale.getlocale()[1] except ValueError: continue # not in table normalized = enc.replace("ISO", "ISO-").
replace("_", "-").
replace("euc", "EUC-").
replace("big5", "big5-").upper() assert normalized == locale.nl_langinfo(locale.CODESET)After my change it does—the encoding returned from getlocale() is the one actually being used by glibc. It fails dramatically on earlier versions of Python (for example on the en_IN example from #29571.) I don't understand why Python needs to editorialize whatever choices libc or the system administrator has made.
Your program essentially tests what alias is configured on your particular system. It will fail on older systems (with a different or no version of SUPPORTED), it will fail on systems that do not have all locales installed, it will fail on systems that use the X.org aliases table as basis rather than some list of supported locales of glibc, or custom alias tables.
What we want in Python is a consistent mapping of aliases to locales across all (Unix based) Python installations, just like what we have for encoding aliases and those mappings should be taken from a support alias database, not a list of default installations on some glibc version.
Also note that a lot of these discussions are really academic, since locales should always be specified with encoding.
While Unix gravitates to UTF-8 for all system related things, users still use other encodings a lot for their daily operations, as you can see in the X.org aliases file.
This is why defaulting to UTF-8 for locales (as e.g. is done for many locales in the glibc default installs) is not a good idea. Locales affect user work products. What's fine for command line interfacing or piping, is not necessarily for fine for e.g. documents created by users.
So to answer your question: No, I don't believe that SUPPORTED has any authority for our purposes and thus don't think that the program can be considered a valid test case.
The SUPPORTED file can server as extra resource for fixing bugs in the table, but nothing more.
Is getlocale() expected to return something different from the underlying C locale?
getlocale() will return whatever is currently configured via setlocale().
Of course, it can return something different from what some glibc SUPPORTED lists as default installation encoding, if you don't provide the encoding when using setlocale(), but it will always default to the same locale and encoding on all platforms where you run Python.
In fact, why have this table at all instead of using nl_langinfo to return the encoding for the current locale?
The table is meant to normalize locale names and enrich them with default encodings from a well known database of such aliases, where necessary. As mentioned above the locale setting should ideally include the encoding as well, so that any such guesses are not necessary.
Regarding nl_langinfo():
nl_langinfo() will only work if you have called setlocale() already, since a process always starts up in the C locale without this call.
If you don't have a problem with calling setlocale() for testing the default locale settings (e.g. Python is not embedded, you don't have other threads running, no APIs which use locale information called yet, setlocale() was already called to setup the locale, etc.), you can use the approach taken by getpreferredencoding(), which is to temporarily set the locale to the default.
Going forward, I think that the following changes make sense:
from ISO8859-1 to ISO8859-15 (the -15 version adds the Euro sign)
casing changes e.g. 'zh_CN.gb2312' to 'zh_CN.GB2312'
fixes which undo removal of modifiers such as 'uz_uz@cyrillic' -> 'uz_UZ.UTF-8' to 'uz_UZ.UTF-8@cyrillic'
As for the other changes: please undo them and also revert the unconditional use of glibc mappings overriding the X.org ones, as mentioned earlier in the thread.
We can readd some of the modifications later on if there's evidence that they actually do make sense.
Thanks,
Marc-Andre Lemburg eGenix.com
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2017-03-10 16:01
I'm feeling there is something wrong with the current locale design. See issues , , , , .
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2017-03-11 07:55
I'm still confused about what getlocale() is supposed to do. Why do we attempt to return an encoding anyway if the underlying setlocale call doesn't return one? Is getlocale() not supposed to a simple wrapper over the C locale? If not, how is one supposed to get the encoding associated with the C locale?
The old alias table code meant that the encoding returned from getlocale() could be related to or completely unrelated to the actual C locale. Misunderstanding this results in issues like #29571.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2017-03-17 21:30
The main purpose of the alias table is to support normalization and this is used for getdefaultencoding() which was created to be able to determine the default encoding based on what X.org uses as default without doing temporary setlocale() tricks.
Now, normalization also happens when passing a locale value to the underlying setlocale(), mainly to avoid many common bugs due to setlocale() being extremely picky about the locale value. A side effect of this is that normalization will also kick in to add the encoding in case no encoding is given in the parameter.
Note that no normalization is necessary to simply set the configured default locale configured on the system. In such a case, you'd run setlocale('LC_ALL') and get what's configured.
If you run the lib C setlocale() with a locale without encoding, the encoding used by the system entirely on what's configured on the system. The SUPPORTED file only gives a hint at what glibc think it should install per default, but any admin or distributor could change these settings simply by running localedef with some other encoding (charmap in locale speak).
I suppose that we could resolve some of the confusion by adding a parameter to disable this normalization in setlocale().
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2017-03-24 20:18
New changeset df8280838f52d6ec45ba03ef734b0dec8a9c43fb by Benjamin Peterson in branch 'master': bpo-20087: Revert "make the glibc alias table take precedence over the X11 one (#422)" (#713) https://github.com/python/cpython/commit/df8280838f52d6ec45ba03ef734b0dec8a9c43fb
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2017-03-24 22:42
New changeset 02371e0ed1ee82ec73e7d363bcf2ed40cde1397a by Benjamin Peterson in branch 'master': make the glibc alias table take precedence over the X11 one (#422) https://github.com/python/cpython/commit/02371e0ed1ee82ec73e7d363bcf2ed40cde1397a
Author: Licht Takeuchi (licht-t) *
Date: 2018-05-05 13:41
Hi all,
The locale in the latest Ubuntu 18.04 contains en_IL as valid locale, but Python cannot resolve this. This makes test failure in pandas. https://github.com/pandas-dev/pandas/issues/20957
en_IL has significant impact because this is English locale and now supported in the latest Ubuntu. Is there any plan to add only en_IL?
(Note that I've already created the PR. ( https://github.com/python/cpython/pull/6707 ))
(pandas-dev) [pandas] locale -a
C
C.UTF-8
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IL
en_IL.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZM
en_ZM.utf8
en_ZW.utf8
ja_JP.utf8
POSIXAuthor: Serhiy Storchaka (serhiy.storchaka) *
Date: 2018-05-05 15:28
Benjamin's patch did two things: 1) made the glibc alias table taking precedence over the X11 one; 2) updated the alias mapping with new glibc. The first part is controversial, but updating the alias mapping with new glibc is made regularly. PR 6708 updates it with glibc 2.27. This adds 39 new aliases and fixes and .
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2018-05-06 05:46
New changeset cedc9b74202d8c1ae39bca261cbb45d42ed54d45 by Serhiy Storchaka in branch 'master': bpo-20087: Update locale alias mapping with glibc 2.27 supported locales. (ПР-6708) https://github.com/python/cpython/commit/cedc9b74202d8c1ae39bca261cbb45d42ed54d45
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2018-05-06 07:20
New changeset 6049bda21b607acc90bbabcc604997e794e8aee1 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.7': [3.7] bpo-20087: Update locale alias mapping with glibc 2.27 supported locales. (GH-6708) (GH-6713) https://github.com/python/cpython/commit/6049bda21b607acc90bbabcc604997e794e8aee1
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2018-05-06 07:20
New changeset b1c70d0ffbb235def1deab62a744ffd9b5253924 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6': [3.6] bpo-20087: Update locale alias mapping with glibc 2.27 supported locales. (GH-6708) (GH-6714) https://github.com/python/cpython/commit/b1c70d0ffbb235def1deab62a744ffd9b5253924
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2018-05-06 07:51
New changeset a55ac801f749a731250f3c7c1db7d546d22ae032 by Serhiy Storchaka in branch '2.7': [2.7] bpo-20087: Update locale alias mapping with glibc 2.27 supported locales. (GH-6708). (GH-6717) https://github.com/python/cpython/commit/a55ac801f749a731250f3c7c1db7d546d22ae032
Author: Marc-Andre Lemburg (lemburg) *
Date: 2018-05-06 11:38
Thanks, Serhiy.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2021-02-17 12:12
I believe we can close this old issue.
The discussion was certainly a useful one. I guess we should stop updating the alias table automatically and instead add new aliases or change existing ones based on more research and using the X11 files as well as glibc and other resources to help.
History
Date
User
Action
Args
2022-04-11 14:57:56
admin
set
github: 64286
2021-02-17 12:12:03
lemburg
set
status: open -> closed
resolution: fixed
messages: +
stage: patch review -> resolved
2019-03-05 15:26:32
vstinner
unlink
2018-05-06 11:38:05
lemburg
set
messages: +
2018-05-06 07:51:51
serhiy.storchaka
set
messages: +
2018-05-06 07:20:44
serhiy.storchaka
set
messages: +
2018-05-06 07:20:14
serhiy.storchaka
set
messages: +
2018-05-06 07:19:19
serhiy.storchaka
set
pull_requests: + <pull%5Frequest6409>
2018-05-06 05:49:25
miss-islington
set
pull_requests: + <pull%5Frequest6407>
2018-05-06 05:47:25
miss-islington
set
pull_requests: + <pull%5Frequest6406>
2018-05-06 05:46:24
serhiy.storchaka
set
messages: +
2018-05-05 15:28:51
serhiy.storchaka
set
messages: +
2018-05-05 15:22:57
serhiy.storchaka
set
keywords: + patch
stage: test needed -> patch review
pull_requests: + <pull%5Frequest6401>
2018-05-05 13:41:00
licht-t
set
nosy: + licht-t
messages: +
2018-05-05 12:42:29
serhiy.storchaka
link
2018-02-15 10:03:40
serhiy.storchaka
link
2017-03-24 22:42:45
benjamin.peterson
set
messages: +
2017-03-24 20🔞34
benjamin.peterson
set
messages: +
2017-03-19 06🔞12
benjamin.peterson
set
pull_requests: + <pull%5Frequest633>
2017-03-18 06:31:35
serhiy.storchaka
set
pull_requests: - <pull%5Frequest602>
2017-03-17 21:30:17
lemburg
set
messages: +
2017-03-17 21:00:34
larry
set
pull_requests: + <pull%5Frequest602>
2017-03-11 07:55:04
benjamin.peterson
set
messages: +
2017-03-10 16:01:12
serhiy.storchaka
set
messages: +
2017-03-10 15:29:14
lemburg
set
messages: +
2017-03-10 07:37:04
benjamin.peterson
set
messages: +
2017-03-09 13:07:38
serhiy.storchaka
set
messages: +
2017-03-09 12:42:38
lemburg
set
messages: +
2017-03-09 10:47:43
serhiy.storchaka
set
messages: +
2017-03-09 10:41:55
serhiy.storchaka
set
messages: +
2017-03-09 10:23:28
lemburg
set
messages: +
2017-03-09 07:15:30
benjamin.peterson
set
messages: +
2017-03-08 15:33:41
lemburg
set
messages: +
2017-03-08 11:48:32
lemburg
set
messages: +
2017-03-08 11:35:17
lemburg
set
messages: +
2017-03-08 09:37:13
serhiy.storchaka
set
messages: +
2017-03-08 09:25:21
lemburg
set
messages: +
2017-03-08 07:20:53
serhiy.storchaka
set
messages: +
2017-03-08 06:27:06
benjamin.peterson
set
nosy: + benjamin.peterson
messages: +
2017-03-07 18:29:01
lemburg
set
messages: +
2017-03-07 17:23:04
serhiy.storchaka
set
messages: +
2017-03-07 16:53:07
lemburg
set
messages: +
2017-03-06 09:41:12
serhiy.storchaka
link
2017-03-06 09:39:59
serhiy.storchaka
set
stage: test needed
versions: + Python 3.6, Python 3.7, - Python 3.4
2017-03-03 08:06:38
serhiy.storchaka
set
messages: +
2017-03-03 07:48:57
benjamin.peterson
set
pull_requests: + <pull%5Frequest351>
2014-09-30 15:45:29
serhiy.storchaka
set
versions: + Python 3.5, - Python 3.3
2013-12-28 20:46:45
Arfrever
set
nosy: + Arfrever
2013-12-28 09:29:49
serhiy.storchaka
create