Issue 17043: Invalid read in test_codecs (original) (raw)

Issue17043

Created on 2013-01-26 20:41 by skrah, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
decodeunicodeinternal_overflow-2.7.patch serhiy.storchaka,2013-01-26 22:06 review
decodeunicodeinternal_overflow-3.2.patch serhiy.storchaka,2013-01-26 22:06 review
decodeunicodeinternal_overflow-3.3.patch serhiy.storchaka,2013-01-26 22:06 review
decodeunicodeinternal_overflow-3.4.patch serhiy.storchaka,2013-01-26 22:06 review
Messages (5)
msg180709 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-01-26 20:41
Found this in test_codecs running under Valgrind (Python 3.3): test_bug1251300 (test.test_codecs.UnicodeInternalTest) ... ==11511== Invalid read of size 1 ==11511== at 0x44AF37: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6133) ==11511== by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251) ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082) ==11511== by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942) ==11511== by 0x491C38: PyCodec_Decode (codecs.c:403) ==11511== by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129) ==11511== by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023) ==11511== by 0x519A45: bytes_decode (bytesobject.c:2320) ==11511== by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374) ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150) ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433) ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160) ==11511== Address 0x984a7e2 is 0 bytes after a block of size 34 alloc'd ==11511== at 0x4C27972: realloc (vg_replace_malloc.c:525) ==11511== by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881) ==11511== by 0x51B1FA: PyBytes_FromObject (bytesobject.c:2732) ==11511== by 0x51C134: bytes_new (bytesobject.c:2594) ==11511== by 0x42A4E4: type_call (typeobject.c:723) ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082) ==11511== by 0x4843D5: PyEval_EvalFrameEx (ceval.c:4282) ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150) ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433) ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160) ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433) ==11511== by 0x538EF8: function_call (funcobject.c:633) ==11511== _PyUnicode_DecodeUnicodeInternal (s=0x984a7e0 "", size=, errors=0x0) at Objects/unicodeobject.c:6133 6133 ((char *) &uch)[2] = s[2]; ================================================================== ==11511== ==11511== Debugger has detached. Valgrind regains control. We continue. ==11511== Invalid read of size 1 ==11511== at 0x44AF3E: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6134) ==11511== by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251) ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082) ==11511== by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942) ==11511== by 0x491C38: PyCodec_Decode (codecs.c:403) ==11511== by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129) ==11511== by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023) ==11511== by 0x519A45: bytes_decode (bytesobject.c:2320) ==11511== by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374) ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150) ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433) ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160) ==11511== Address 0x984a7e3 is 1 bytes after a block of size 34 alloc'd ==11511== at 0x4C27972: realloc (vg_replace_malloc.c:525) ==11511== by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881) ==11511== by 0x51B1FA: PyBytes_FromObject (bytesobject.c:2732) ==11511== by 0x51C134: bytes_new (bytesobject.c:2594) ==11511== by 0x42A4E4: type_call (typeobject.c:723) ==11511== by 0x5093F6: PyObject_Call (abstract.c:2082) ==11511== by 0x4843D5: PyEval_EvalFrameEx (ceval.c:4282) ==11511== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150) ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433) ==11511== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160) ==11511== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433) ==11511== by 0x538EF8: function_call (funcobject.c:633) ==11511== Loaded symbols for /usr/lib/gconv/ISO8859-9.so _PyUnicode_DecodeUnicodeInternal (s=0x8295790 "", size=, errors=0x0) at Objects/unicodeobject.c:6134 6134 ((char *) &uch)[3] = s[3];
msg180711 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-01-26 21:02
Same in test_codeccallbacks: test_badhandlerresults (test.test_codeccallbacks.CodecCallbackTest) ... ==11604== Invalid read of size 1 ==11604== at 0x44AF37: _PyUnicode_DecodeUnicodeInternal (unicodeobject.c:6133) ==11604== by 0x4DEB5C: unicode_internal_decode (_codecsmodule.c:251) ==11604== by 0x5093F6: PyObject_Call (abstract.c:2082) ==11604== by 0x47D7F2: PyEval_CallObjectWithKeywords (ceval.c:3942) ==11604== by 0x491C38: PyCodec_Decode (codecs.c:403) ==11604== by 0x459D7D: PyUnicode_Decode (unicodeobject.c:3129) ==11604== by 0x45A287: PyUnicode_FromEncodedObject (unicodeobject.c:3023) ==11604== by 0x519A45: bytes_decode (bytesobject.c:2320) ==11604== by 0x484AB8: PyEval_EvalFrameEx (ceval.c:4374) ==11604== by 0x485ACB: PyEval_EvalFrameEx (ceval.c:4150) ==11604== by 0x486779: PyEval_EvalCodeEx (ceval.c:3433) ==11604== by 0x4859CA: PyEval_EvalFrameEx (ceval.c:4160) ==11604== Address 0xfa1f8a2 is 0 bytes after a block of size 34 alloc'd ==11604== at 0x4C27972: realloc (vg_replace_malloc.c:525) ==11604== by 0x51AC34: _PyBytes_Resize (bytesobject.c:2881) ==11604== by 0x51C338: PyBytes_DecodeEscape (bytesobject.c:495) ==11604== by 0x56E871: ast_for_expr (ast.c:3837) ==11604== by 0x570562: ast_for_testlist (ast.c:1106) ==11604== by 0x56E859: ast_for_expr (ast.c:1881) ==11604== by 0x570562: ast_for_testlist (ast.c:1106) ==11604== by 0x56E859: ast_for_expr (ast.c:1881) ==11604== by 0x5715C4: ast_for_stmt (ast.c:3302) ==11604== by 0x5724F8: ast_for_suite (ast.c:3086) ==11604== by 0x5715E3: ast_for_stmt (ast.c:3305) ==11604== by 0x5724F8: ast_for_suite (ast.c:3086) _PyUnicode_DecodeUnicodeInternal (s=0xfa1f8a0 "", size=, errors= 0xf652fa0 "test.badhandler") at Objects/unicodeobject.c:6133 6133 ((char *) &uch)[2] = s[2]; [...] _PyUnicode_DecodeUnicodeInternal (s=0xfa1f8a0 "", size=, errors= 0xf652fa0 "test.badhandler") at Objects/unicodeobject.c:6134 6134 ((char *) &uch)[3] = s[3];
msg180716 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-26 22:06
Here are patches for all 4 versions.
msg181437 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-02-05 14:07
Ping.
msg181610 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-02-07 14:30
New changeset 498b54e0e856 by Serhiy Storchaka in branch '2.7': Issue #17043: The unicode-internal decoder no longer read past the end of http://hg.python.org/cpython/rev/498b54e0e856 New changeset 0f1c2e2b6bc2 by Serhiy Storchaka in branch '3.2': Issue #17043: The unicode-internal decoder no longer read past the end of http://hg.python.org/cpython/rev/0f1c2e2b6bc2 New changeset fec2976c8503 by Serhiy Storchaka in branch '3.3': Issue #17043: The unicode-internal decoder no longer read past the end of http://hg.python.org/cpython/rev/fec2976c8503 New changeset eb0370d4686c by Serhiy Storchaka in branch 'default': Issue #17043: The unicode-internal decoder no longer read past the end of http://hg.python.org/cpython/rev/eb0370d4686c
History
Date User Action Args
2022-04-11 14:57:41 admin set github: 61245
2013-02-09 20:45:31 serhiy.storchaka set status: open -> closedresolution: fixedstage: patch review -> resolved
2013-02-07 14:30:47 python-dev set nosy: + python-devmessages: +
2013-02-05 14:07:29 serhiy.storchaka set messages: +
2013-01-31 14:38:51 serhiy.storchaka set assignee: serhiy.storchaka
2013-01-26 22:07:47 serhiy.storchaka set versions: + Python 2.7, Python 3.2, Python 3.4nosy: + ezio.melotticomponents: + Interpreter Core, Unicodetype: behaviorstage: patch review
2013-01-26 22:06:44 serhiy.storchaka set files: + decodeunicodeinternal_overflow-2.7.patch, decodeunicodeinternal_overflow-3.2.patch, decodeunicodeinternal_overflow-3.3.patch, decodeunicodeinternal_overflow-3.4.patchkeywords: + patchmessages: +
2013-01-26 21:02:36 skrah set messages: +
2013-01-26 20:41:15 skrah create