[Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale (original) (raw)

INADA Naoki songofacandy at gmail.com
Mon May 8 04🔞15 EDT 2017

On platforms where they would have no effect (e.g. Mac OS X, iOS, Android, Windows) these preprocessor variables would always be undefined.

Why --with[out]-c-locale-coercion have no effect on macOS, iOS and Android? On these three, we know the system encoding is UTF-8, so we never interpreted the C locale as meaning "ascii" in the first place. On Android, locale coercion fixes readline. Do you mean locale coercion happen always regardless this configuration option? Right, the change for Android is that we switch to calling 'setlocale(LCALL, "C.UTF-8")' during interpreter startup instead of 'setlocale(LCALL, "")'. That change is guarded by "#ifdef ANDROID", rather than either of the new conditionals. On macOS, LCALL=C python doesn't make Python's stdio to ascii:surrogateescape? Similar to Android, CPython itself is hardcoded to assume UTF-8 on Mac OS X, since that's a platform API guarantee that users can't change. Even so, locale coercion may fix libraries like readline, curses. While C locale is less common on macOS, I don't understand any reason to disable it on macOS. My understanding is that other libraries and applications also automatically use UTF-8 for system interfaces on Mac OS X and iOS. It could be that that understanding is wrong, and locale coercion would provide a benefit there as well. (Checking the draft implementation, it turns out I haven't actually implemented the configure logic to make those config settings platform dependent yet - they're currently only undefined on Windows by default, since that doesn't use the autotools based build system)

I tried Python 3.6 on macOS 10.11 El Capitan.

$ LANG=C python3 -c 'import locale; print(locale.getpreferredencoding())' US-ASCII

And interactive shell (which uses readline by default) doesn't accept non-ASCII input anymore. https://www.dropbox.com/s/otshuzhnw7a71n5/macos-c-locale-readline.gif?dl=0

I think many problems with C locale are same on macOS too. So I don't think no special casing is required on macOS.

Regards,

More information about the Python-Dev mailing list