Issue 8374: Some locales are unsupported (original) (raw)

Issue8374

Created on 2010-04-11 21:20 by ubuntujenkins, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (20)
msg102891 - (view)	Author: Luke Jennings (ubuntujenkins)	Date: 2010-04-11 21:20
In the locale module there are some locales that are not supported these the ones that I am aware of are nl_AW, sr_RS sr_ME. This information was due to a project that captures screenshots in different languages and we have to retrieve the language code. Related to the origin of the bug https://bugs.edge.launchpad.net/quickshot/+bug/554861 . If any more information is required please let me know.
msg102892 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2010-04-11 21:38
Hello Not a local expert here, but since this module relies on the underlying libc locale support. Do other programs work correctly with this locale? Apart from that, your program needs to catch and handle exceptions anyway. Martin, I’m making you nosy, since you’re listed as locale area expect in the maintainers file. Hope it’s okay to do so. Regards
msg102893 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2010-04-11 21:43
Sorry for sending this too fast. The beginning should read: Not a locale expert here, but since this module relies on the underlying libc locale support, the bug is not Python-specific.
msg102894 - (view)	Author: Luke Jennings (ubuntujenkins)	Date: 2010-04-11 21:47
Other programs do work with the local. I am working on dealing with exceptions, I am rather new to programing and thought it would also be good to try and get this fixed in the original module.
msg102895 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2010-04-11 21:49
What I’m saying is that the original module is not a bug. I’ll let Martin confirm or infirm :) Regards
msg102896 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2010-04-11 21:52
This doesn't look like a bug. You must install a locale before it gets available to Python. Furthermore, according to the original traceback ( http://launchpadlibrarian.net/43012745/quickshot-latin.log ), the locale module isn't even involved.
msg102897 - (view)	Author: Luke Jennings (ubuntujenkins)	Date: 2010-04-11 21:58
Sorry for the confusion but that bug report has two very similar problems to it if you look at https://bugs.edge.launchpad.net/quickshot/+bug/554861/comments/7 and https://bugs.edge.launchpad.net/quickshot/+bug/554861/comments/6 . I think that is calling the locale module. I apologize if i have put this down to the wrong thing I am new to python.
msg102900 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2010-04-11 22:02
Actually, my diagnosis was wrong. The locale module contains a list of locale aliases and some of the Serbian ones are missing. A recent locale.alias file from X11 is more complete.
msg102901 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2010-04-11 22:13
getdefaultlocale is inherently unmaintainable, and shouldn't be used by applications. I wish it was removed from Python (but unfortunately, too many people got tricked into believing that it does something useful). That said, if anybody feels like updating the tables from the various data sources, please go ahead.
msg102902 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2010-04-11 22:15
Can we tell about getdefaultlocale’s uselessness in the docs?
msg102903 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2010-04-11 22:23
> Can we tell about getdefaultlocale’s uselessness in the docs? I haven't quite understood what people want to use that function for. If we knew the typical use cases, we could make recommendations what they should use instead. One use case is to find out the encoding of the locale, in order to encode files. People should use locale.getpreferredencoding instead, or trust that the io module will find out on its own. However, these cases seem to be different.
msg102904 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2010-04-11 22:27
> However, these cases seem to be different. I think it can be used to find out the current language setting of the user, and trigger a third-party translation system (other than e.g. gettext).
msg102905 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2010-04-11 22:27
When the table is updated in trunk, can it be backported to 2.6? It does change a couple of mappings (example here is from sr_CS.* to sr_RS.*).
msg102909 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2010-04-11 23:04
> When the table is updated in trunk, can it be backported to 2.6? With the changes to the encodings for some of the locales (e.g. 'ru'), I would advise against such a backport. This also demonstrates one fundamental flaw of the approach: even if X11 decides that 'ru' should be using "UTF-8", some local installations will certainly differ. Likewise when they decided that "ru" uses iso-8859-5.
msg102910 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2010-04-11 23:08
Ok, the fix (updated tables) has been committed in trunk (r79977) and py3k (r79978). I suppose Ubuntu can backport the patch if they want their Python to be in sync with their own locales. (even better, distributions should include the regeneration of locale aliases from their own locale.alias in their build scripts)
msg102923 - (view)	Author: Marc-Andre Lemburg (lemburg) *	Date: 2010-04-12 08:19
Martin v. Löwis wrote: > > Martin v. Löwis <martin@v.loewis.de> added the comment: > > getdefaultlocale is inherently unmaintainable, and shouldn't be used by applications. I wish it was removed from Python (but unfortunately, too many people got tricked into believing that it does something useful). I disagree on that. getdefaultlocale() provides a way to access the default locale (and encoding) on a platform without requiring a call to setlocale(LC_ALL, "") which would affect the currently running application and is not thread-safe (just like most other locale C APIs in the C lib). getpreferredencoding() just provides encoding information. It doesn't provide locale information, so I don't see how that could be used as replacement for getdefaultlocale(). The table needed for normalizing the various locale strings used in the LANG environment variable (and others) needs to be updated every now and then, but that's natural for a resource that changes based on what people use locales for in real life. > That said, if anybody feels like updating the tables from the various data sources, please go ahead. I last updated the table in 2008. Will do that again this week.
msg102926 - (view)	Author: Marc-Andre Lemburg (lemburg) *	Date: 2010-04-12 08:55
M.-A. Lemburg wrote: >> That said, if anybody feels like updating the tables from the various data sources, please go ahead. > > I last updated the table in 2008. Will do that again this week. I saw that Antoine already did this for me: thanks, Antoine ! Could you tell me which X11 version you used ? I've modified the generation script to also output the added aliases and would like to list them as well.
msg102942 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2010-04-12 12:48
> Could you tell me which X11 version you used ? I've modified the > generation script to also output the added aliases and would like > to list them as well. It's the file supplied on my Mandriva system, from libx11-1.2.2-2mdv2010.0.src.rpm
msg102944 - (view)	Author: Jeroen Ruigrok van der Werven (asmodai) *	Date: 2010-04-12 13:05
That nl_AW is an interesting one. Aruba is part of the Kingdom of the Netherlands, so in essence it should follow nl_NL. From what I can see that locale seems to be limited to (certain) Linux systems. I'll raise the issue with the CLDR guys as well, since I wonder if this warrants its own locale or not (some monetary and number formatting). But I think the outcome will be to add a nl_AW as well in similar vein as to what bo_CN does.
msg102969 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2010-04-12 20:09
> getdefaultlocale() provides a way to access the default locale > (and encoding) on a platform without requiring a call to > setlocale(LC_ALL, "") That's what it's meant to do, but this is not what it actually does. In fact, there is no way of determining the locale's encoding without the help o fthe C library. As for "default locale": this is a flawed notion in itself. If I use one locale for messages, and the other for date representation - which one should be the "default" locale? Also, "default" in what respect? > getpreferredencoding() just provides encoding information. It doesn't > provide locale information, so I don't see how that could be used > as replacement for getdefaultlocale(). Many applications use getdefaultlocale()[1], i.e. ignore the locale part of the result. They can easily be rewritten to use getpreferredencoding. > The table needed for normalizing the various locale strings used in > the LANG environment variable (and others) needs to be updated every > now and then, but that's natural for a resource that changes based > on what people use locales for in real life. However, it also changes based on local administrator preferences - something that the implementation does not take into account at all. It may also depend on OS vendor preferences, which is also not considered in the implementation. IOW, it returns bogus results.