| msg235177 - (view) |
Author: paul (pkt) |
Date: 2015-02-01 13:59 |
| # static PyObject * # ascii_escape_unicode(PyObject *pystr) # { # ... # # input_chars = PyUnicode_GET_LENGTH(pystr); # input = PyUnicode_DATA(pystr); # kind = PyUnicode_KIND(pystr); # # /* Compute the output size */ # for (i = 0, output_size = 2; i < input_chars; i++) { # Py_UCS4 c = PyUnicode_READ(kind, input, i); # if (S_CHAR(c)) # output_size++; # else { # switch(c) { # ... # default: # 1 output_size += c >= 0x10000 ? 12 : 6; # ... # # 2 rval = PyUnicode_New(output_size, 127); # # 1. if c is \uFFFF then output_size += 6. There are no overflow checks on this # variable, so we can overflow it with a sufficiently long (2**32/6+1 chars) # string # 2. rval buffer is too small to hold the result # # Crash: # ------ # # Breakpoint 3, ascii_escape_unicode (pystr='...') at /home/p/Python-3.4.1/Modules/_json.c:198 # 198 rval = PyUnicode_New(output_size, 127); # (gdb) print output_size # $9 = 4 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x4057888f in ascii_escape_unichar (c=65535, # output=0x40572358 "...", # chars=19624) at /home/p/Python-3.4.1/Modules/_json.c:155 # 155 output[chars++] = Py_hexdigits[(c >> 8) & 0xf]; # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # from _json import encode_basestring_ascii as enc s="\uffff"*int((2**32)/6+1) enc(s) |
|
|
| msg235213 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2015-02-01 23:02 |
| New changeset 8699b3085db3 by Benjamin Peterson in branch '3.3': fix possible overflow in encode_basestring_ascii (closes #23369) https://hg.python.org/cpython/rev/8699b3085db3 New changeset 4f47509d7417 by Benjamin Peterson in branch '3.4': merge 3.3 (#23369) https://hg.python.org/cpython/rev/4f47509d7417 New changeset 02aeca4974ac by Benjamin Peterson in branch 'default': merge 3.4 (#23369) https://hg.python.org/cpython/rev/02aeca4974ac |
|
|
| msg235255 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2015-02-02 13:09 |
| "\uffff"*((2**32)//6 + 1) is calculated at compile time. This requires much memory and can cause swapping. May be this was a cause of failing tests on some buildbots: http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.x/builds/2623/steps/test/logs/stdio http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.4/builds/749/steps/test/logs/stdio Traceback (most recent call last): File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/runpy.py", line 170, in _run_module_as_main "__main__", mod_spec) File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/test/__main__.py", line 3, in regrtest.main_in_temp_cwd() File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/test/regrtest.py", line 1564, in main_in_temp_cwd main() File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/test/regrtest.py", line 738, in main raise Exception("Child error on {}: {}".format(test, result[1])) Exception: Child error on test_json: Exit code -9 *** [buildbottest] Error code 1 At least my computer hanged on first run of this test. To prevent computing this string constant at compile time you can use a variable. And '\x00' can be used instead of '\uffff', it needs less memory. |
|
|
| msg235298 - (view) |
Author: Roundup Robot (python-dev) |
Date: 2015-02-02 22:47 |
| New changeset 5c730d30ffbc by Benjamin Peterson in branch '3.3': reduce memory usage of test (closes #23369) https://hg.python.org/cpython/rev/5c730d30ffbc |
|
|
| msg272156 - (view) |
Author: tehybel_ (tehybel_) |
Date: 2016-08-08 09:54 |
| I noticed that this is only fixed for python 3.3 and 3.4, not for 2.7. Is that intentional? If so, why? |
|
|
| msg272623 - (view) |
Author: Roundup Robot (python-dev) |
Date: 2016-08-13 23:48 |
| New changeset 6fa0ebfdc136 by Benjamin Peterson in branch '2.7': fix possible overflow in encode_basestring_ascii (#23369) https://hg.python.org/cpython/rev/6fa0ebfdc136 |
|
|