| msg297587 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2017-07-03 13:57 |
| One example: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/957/steps/test/logs/stdio 0:08:55 [ 91/406/2] test_c_locale_coercion failed -- running: test_importlib (41 sec) skipped 'No C-with-UTF-8 locale available' test_LC_ALL_set_to_C (test.test_c_locale_coercion.LocaleCoercionTests) ... test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) ... test_PYTHONCOERCECLOCALE_set_to_warn (test.test_c_locale_coercion.LocaleCoercionTests) ... test_PYTHONCOERCECLOCALE_set_to_zero (test.test_c_locale_coercion.LocaleCoercionTests) ... test_test_PYTHONCOERCECLOCALE_not_set (test.test_c_locale_coercion.LocaleCoercionTests) ... ====================================================================== FAIL: test_LC_ALL_set_to_C (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='', PYTHONCOERCECLOCALE=None) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_c_locale_coercion.py", line 309, in _check_c_locale_coercion coercion_expected) File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_c_locale_coercion.py", line 202, in _check_child_encoding_details self.assertEqual(encoding_details, expected_details) AssertionError: {'fsencoding': 'iso8859-1', 'stdin_info': 'iso8859-1:surr[140 chars] 'c'} != {'fsencoding': 'ascii', 'stdin_info': 'ascii:surrogateesc[124 chars] 'c'} - {'fsencoding': 'iso8859-1', ? ^^^^^^^^ + {'fsencoding': 'ascii', ? +++ ^ 'lang': '', 'lc_all': 'c', 'lc_ctype': '', - 'stderr_info': 'iso8859-1:backslashreplace', ? ^^^^^^^^ + 'stderr_info': 'ascii:backslashreplace', ? +++ ^ - 'stdin_info': 'iso8859-1:surrogateescape', ? ^^^^^^^^ + 'stdin_info': 'ascii:surrogateescape', ? +++ ^ - 'stdout_info': 'iso8859-1:surrogateescape'} ? ^^^^^^^^ + 'stdout_info': 'ascii:surrogateescape'} ? +++ ^ |
|
|
| msg297592 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2017-07-03 14:21 |
| This should be a fairly straightforward test-only fix: updating the start of https://github.com/python/cpython/blob/master/Lib/test/test_c_locale_coercion.py to set C_LOCALE_STREAM_ENCODING to 'iso8859-1' when "sys.platform.startswith('aix')". |
|
|
| msg297593 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2017-07-03 14:23 |
| I also belatedly checked your list at https://unicodebook.readthedocs.io/operating_systems.html#locale-encoding, and confirmed that AIX is the only anomalous case you'd previously identified (aside from the odd nominally-ASCII-but-not-really behaviour on FreeBSD, Mac OS X, and Solaris). |
|
|
| msg297701 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2017-07-05 05:40 |
| Removing the "easy" tag, as I realised this isn't as clearcut a change as I first thought. Specifically, I'm wondering if in addition to changing the expectations for this particular test, we should *also* be disabling locale coercion on AIX in general. After all, latin-1 already round trips just as well as UTF-8+surrogateescape does, and IBM don't seem to have any plans in progress to change the default environment on AIX away from latin-1. |
|
|
| msg297702 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2017-07-05 05:48 |
| Note that as far as I'm aware, AIX doesn't actually provide any of our coercion target locales yet, so the question in my previous comment is about how we want 3.7.x releases to behave on AIX if IBM *do* introduce a C.UTF-8 locale. The main argument I see in favour of leaving locale coercion enabled is that the latin-1 default will cope with UTF-8 encoded data from Python 3.7+ applications (even if it displays them incorrectly), but may raise UnicodeEncodeError if an application attempts to create arbitrary Unicode filenames, or write arbitrary unicode data to the standard streams. |
|
|
| msg298395 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2017-07-15 12:51 |
| New changeset f0b6a261bb8cbede42296e0cc70956fb3b9a6cf7 by Nick Coghlan in branch 'master': bpo-30836: fix test_c_locale_coercion on AIX (GH-2713) https://github.com/python/cpython/commit/f0b6a261bb8cbede42296e0cc70956fb3b9a6cf7 |
|
|
| msg298396 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2017-07-15 12:52 |
| This particular test passed on the custom AIX buildbot prior to merging (other tests still have issues on AIX, though, so that buildbot remains red overall). |
|
|
| msg298407 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-07-15 22:26 |
| Thanks for the fix :-) |
|
|