Issue 17173: uses of locale-dependent ctype functions (original) (raw)

This issue has been migrated to GitHub: https://github.com/python/cpython/issues/61375

classification

process

Created on 2013-02-09 20:49 by pitrou, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (5)
msg181759 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2013-02-09 20:49
Grepping through the code reveals we are still using a number of locale-dependent C library functions: Python/mystrtoul.c:102: while (*str && isspace(Py_CHARMASK(*str))) Python/mystrtoul.c:141: while (isspace(Py_CHARMASK(*str))) Python/mystrtoul.c:269: while (*str && isspace(Py_CHARMASK(*str))) Python/formatter_unicode.c:404: while (pos<end && isdigit(PyUnicode_READ_CHAR(s, pos))) Python/ast.c:3764: if (isalpha(quote)) { Python/dynload_aix.c:144: while (isdigit(Py_CHARMASK(*message[i]))) message[i]++ ; Objects/longobject.c:2011: while (*str != '\0' && isspace(Py_CHARMASK(*str))) Objects/longobject.c:2255: while (*str && isspace(Py_CHARMASK(*str))) Modules/getaddrinfo.c:228: if (! isdigit(*q)) Modules/_sre.c:153:#define SRE_LOC_IS_DIGIT(ch) (!((ch) & ~255) ? isdigit((ch)) : 0) Modules/_sre.c:154:#define SRE_LOC_IS_SPACE(ch) (!((ch) & ~255) ? isspace((ch)) : 0) Modules/_sre.c:156:#define SRE_LOC_IS_ALNUM(ch) (!((ch) & ~255) ? isalnum((ch)) : 0) Modules/binascii.c:1138: if (isdigit(c)) Modules/_decimal/libmpdec/io.c:153: if (!isdigit((uchar)*s)) Modules/_decimal/libmpdec/io.c:157: if (!isdigit((uchar)*(s+1))) Modules/_decimal/libmpdec/io.c:159: isdigit((uchar)*(s+2)))) Modules/_decimal/libmpdec/io.c:185: while (isdigit((uchar)*s)) Modules/_decimal/libmpdec/io.c:825: if (isdigit((uchar)*cp)) { Modules/_decimal/libmpdec/io.c:847: if (!isdigit((uchar)*cp)) { Modules/_decimal/libmpdec/io.c:1087: assert(isdigit((uchar)*dp)); Modules/_decimal/libmpdec/io.c:1089: while (isdigit((uchar)*dp)) { Modules/_decimal/libmpdec/io.c:1244: if (isupper((uchar)type)) { Modules/_struct.c:1273: if (isspace(Py_CHARMASK(c))) Modules/_struct.c:1338: if (isspace(Py_CHARMASK(c))) Include/pyctype.h has locale-agnostic counterpart (a bit faster probably, too).
msg181760 - (view)	Author: Raymond Hettinger (rhettinger) *	Date: 2013-02-09 20:57
+1 for fixing this everywhere.
msg181763 - (view)	Author: Stefan Krah (skrah) *	Date: 2013-02-09 21:41
I'm not sure if I'll use pyctype.h in libmpdec: It's still going to be an external project that should be completely identical to the version in the Python tree. libmpdec/io.c is specified to be ASCII only (while handling the Turkish 'I') and is used accordingly in _decimal.c. I think it is impossible to trigger any misbehavior just by using the decimal module.
msg181765 - (view)	Author: Roundup Robot (python-dev)	Date: 2013-02-09 22:29
New changeset 38830281d43b by Antoine Pitrou in branch '3.2': Issue #17173: Remove uses of locale-dependent C functions (isalpha() etc.) in the interpreter. http://hg.python.org/cpython/rev/38830281d43b New changeset c08bcf5302ec by Antoine Pitrou in branch '3.3': Issue #17173: Remove uses of locale-dependent C functions (isalpha() etc.) in the interpreter. http://hg.python.org/cpython/rev/c08bcf5302ec New changeset 10e59553a8de by Antoine Pitrou in branch 'default': Issue #17173: Remove uses of locale-dependent C functions (isalpha() etc.) in the interpreter. http://hg.python.org/cpython/rev/10e59553a8de
msg181766 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2013-02-09 22:31
Done. I haven't touched _decimal, sre, getaddrinfo.c and zlib.