[Python-Dev] Issue #11022: locale.getpreferredencoding() must not set temporary LC_CTYPE (original) (raw)

Victor Stinner victor.stinner at gmail.com
Tue Jun 5 02:05:26 CEST 2012

Hi,

I would like to know if it is too late (or not) to change the behaviour of open() for text files (TextIOWrapper). Currently, it calls locale.getpreferredencoding() to get the locale encoding by default. It is convinient and I like this behaviour... except that it changes temporary the LC_CTYPE locale to get the user prefered encoding instead of using the current encoding. Python 3 does already uses the user preferred encoding as the current encoding at startup. Changing temporary the current encoding to the user preferred encoding is useless and dangerous (may cause issues in multithreaded applications). Setting the current locale using locale.setlocale() does not affect TextIOWrapper, it's also surprising.

The change is just to replace locale.getpreferredencoding() by locale.getpreferredencoding(False) in the io module.

Can I change this behaviour (before the first beta) in Python 3.3?

See the issue #11022 (and maybe also #6203) for the discussion on this change.

--

Leaving LC_CTYPE unchanged (use the "C" locale, which is ASCII in most cases) at Python startup would be a major change in Python 3. I don't want to do that. You would see a lot of mojibake in your GUIs and get a lot of ugly surrogate characters in filenames because of the PEP 393.

Setting the LC_CTYPE to the user preferred encoding is just very convinient and helps Python to speak to the user though the console, to the filesystem, to pass arguments on a command line of a subprocess, etc. For example, you cannot pass non-ASCII characters to a subprocess, characters written by the user in your GUI, if your current LC_CTYPE locale is C (ASCII): you get an Unicode encode error.

Victor

More information about the Python-Dev mailing list