Issue 9233: json.load failure when C optimizations aren't built (original) (raw)
Created on 2010-07-12 15:47 by fdrake, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (10)
Author: Fred Drake (fdrake) 
Date: 2010-07-12 15:47
The json decoder doesn't pass tests when the C optimizations in the _json module aren't available. The actual test failures appear fairly superficial, but aren't the limit of the problems.
Empty objects are converted to empty lists instead of empty dictionaries when _json isn't available; test attached.
Issue 5723 notes that the pure-Python implementation isn't tested; fixing that might have avoided these problems.
Test failures from Python 2.6.5:
====================================================================== ERROR: test_c_encode_basestring_ascii (json.tests.test_encode_basestring_ascii.TestEncodeBaseStringAscii)
Traceback (most recent call last): File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_encode_basestring_ascii.py", line 29, in test_c_encode_basestring_ascii self._test_encode_basestring_ascii(json.encoder.c_encode_basestring_ascii) File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_encode_basestring_ascii.py", line 32, in _test_encode_basestring_ascii fname = encode_basestring_ascii.name AttributeError: 'NoneType' object has no attribute 'name'
====================================================================== ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
Traceback (most recent call last): File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring self._test_scanstring(json.decoder.c_scanstring) File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring scanstring('"z\ud834\udd20x"', 1, None, True), TypeError: 'NoneType' object is not callable
====================================================================== FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
Traceback (most recent call last): File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_speedups.py", line 13, in test_encode_basestring_ascii self.assertEquals(encoder.encode_basestring_ascii.module, "_json") AssertionError: 'json.encoder' != '_json'
====================================================================== FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
Traceback (most recent call last): File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_speedups.py", line 9, in test_scanstring self.assertEquals(decoder.scanstring.module, "_json") AssertionError: 'json.decoder' != '_json'
Test failures from 2.7:
====================================================================== ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
Traceback (most recent call last): File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring self._test_scanstring(json.decoder.c_scanstring) File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring scanstring('"z\ud834\udd20x"', 1, None, True), TypeError: 'NoneType' object is not callable
====================================================================== FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
Traceback (most recent call last): File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_speedups.py", line 12, in test_encode_basestring_ascii self.assertEquals(encoder.encode_basestring_ascii.module, "_json") AssertionError: 'json.encoder' != '_json'
====================================================================== FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
Traceback (most recent call last): File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_speedups.py", line 8, in test_scanstring self.assertEquals(decoder.scanstring.module, "_json") AssertionError: 'json.decoder' != '_json'
Author: Fred Drake (fdrake)
Date: 2010-07-12 16:51
The attached test only fails for Python 2.7; removing Python 2.6 from this issue.
Author: Bob Ippolito (bob.ippolito) *
Date: 2011-01-27 00:48
For what it's worth, I don't think any of the failures mentioned here are in the current version of simplejson. There aren't any tests that check the speedup module name, the speedup tests are skipped when speedups are not detected, and the empty json object bug was fixed in June 2009.
http://code.google.com/p/simplejson/issues/detail?id=57
Author: Alex Gaynor (alex) *
Date: 2011-01-28 22:03
Just a note from downstream with PyPy: we've cherry-picked the commit Bob linked into our copy of the 2.7 stdlib since we don't have an _json (yet).
Author: Ezio Melotti (ezio.melotti) *
Date: 2011-04-12 12:39
I ran the tests again on 2.7 and got 3 failures:
====================================================================== ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
Traceback (most recent call last): File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring self._test_scanstring(json.decoder.c_scanstring) File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring scanstring('"z\ud834\udd20x"', 1, None, True), TypeError: 'NoneType' object is not callable
====================================================================== FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
Traceback (most recent call last): File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_speedups.py", line 14, in test_encode_basestring_ascii self.assertEqual(encoder.encode_basestring_ascii.module, "_json") AssertionError: 'json.encoder' != '_json'
====================================================================== FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
Traceback (most recent call last): File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_speedups.py", line 11, in test_scanstring self.assertTrue(decoder.scanstring is decoder.c_scanstring) AssertionError: False is not true
The attached patch checks if _json is importable and skip the tests if it's not.
Author: Roundup Robot (python-dev) 
Date: 2011-04-12 13:07
New changeset 500063f6ae5a by Ezio Melotti in branch '2.7': #9233: skip _json-specific tests when _json is not available. http://hg.python.org/cpython/rev/500063f6ae5a
Author: Terry J. Reedy (terry.reedy) *
Date: 2011-04-12 18:59
Do those new check finish this issue and should it be closed? If not, what is left?
Author: Roundup Robot (python-dev)
Date: 2011-04-13 02:38
New changeset d58c63ff5bb2 by Ezio Melotti in branch '2.7': #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available. http://hg.python.org/cpython/rev/d58c63ff5bb2
Author: Roundup Robot (python-dev)
Date: 2011-04-13 04:21
New changeset 36e676a8541d by Ezio Melotti in branch '3.1': #9233: skip _json-specific tests when _json is not available. http://hg.python.org/cpython/rev/36e676a8541d
New changeset 7019fc1a9663 by Ezio Melotti in branch '3.1': #9233: Fix json to work properly even when _json is not available. http://hg.python.org/cpython/rev/7019fc1a9663
New changeset a220458179ed by Ezio Melotti in branch '3.1': #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available. http://hg.python.org/cpython/rev/a220458179ed
Author: Ezio Melotti (ezio.melotti) *
Date: 2011-04-13 04:25
Fixed in 2.7 and 3.1 and ported to 3.2 (b279611146d7) and 3.3 (e8e3f2b72a32). In 3.1 json was completely broken without _json, so I fixed that too. I also did some minor cleanup in ec6d881f5b02.
History
Date
User
Action
Args
2022-04-11 14:57:03
admin
set
github: 53479
2011-04-13 04:25:47
ezio.melotti
set
status: open -> closed
versions: + Python 3.1, Python 3.2, Python 3.3
messages: +
resolution: fixed
stage: needs patch -> resolved
2011-04-13 04:21:52
python-dev
set
messages: +
2011-04-13 02:38:17
python-dev
set
messages: +
2011-04-12 18:59:14
terry.reedy
set
nosy: + terry.reedy
messages: +
2011-04-12 13:09:01
ezio.melotti
set
assignee: ezio.melotti
versions: - Python 2.6
2011-04-12 13:07:28
python-dev
set
nosy: + python-dev
messages: +
2011-04-12 12:39:29
ezio.melotti
set
files: + issue9233.diff
messages: +
2011-01-28 22:03:22
alex
set
nosy: + alex
messages: +
2011-01-27 00:48:50
bob.ippolito
set
nosy:fdrake, bob.ippolito, dstanek, ezio.melotti
messages: +
2011-01-26 17:54:34
dstanek
set
nosy: + dstanek
2010-07-12 16:51:36
fdrake
set
messages: +
2010-07-12 15:49:18
ezio.melotti
set
nosy: + bob.ippolito, ezio.melotti
stage: needs patch
2010-07-12 15:47:44
fdrake
create