[Python-Dev] Status of json (simplejson) in cpython (original) (raw)

Vinay Sajip vinay_sajip at yahoo.co.uk
Sat Apr 16 11:50:25 CEST 2011

Sandro Tosi <sandro.tosi gmail.com> writes:

The version we have in cpython of json is simplejson 2.0.9 highly patched (either because it was converted to py3k, and because of the normal flow of issues/bugfixes) while upstream have already released 2.1.13 .

I think you mean 2.1.3?

Their 2 roads had diverged a lot, and since this blocks any further update of cpython's json from upstream, I'd like to close this gap.

This isn't exactly an easy task, and this email is more about a brainstorming on the ways we have to achieve the goal: being able to upgrade json to 2.1.13. Luckily, upstream is receptive for patches, so part of the job is to forward patches written for cpython not already in the upstream code. But how am I going to do this? let's do a brain-dump: - the history goes back at changeset f686aced02a3 (May 2009, wow) when 2.0.9 was merged on trunk - I can navigate from that CS up to tip, and examine the diffs and see if they apply to 2.1.3 and prepare a set of patches to be forwarded - part of those diffs is about py3k conversion, that probably needs to be extended to other part of the upstream code not currently in cpython. For those "new" code parts, do you have some guides about porting a project to py3k? it would be my first time and other than building it and running it with python3 i don't know what to do :) - once (and if :) I reach the point where I've all the relevant patches applied on 2.1.3 what's the next step?

If it is generally considered desirable to maintain some synchrony between simplejson and stdlib json, then since Bob has stated that he no interest in Python 3, it may be better to:

  1. Convert the simplejson codebase so that it runs on both Python 2 and 3 (without running 2to3 on it). Once this is done, if upstream accepts these changes, ongoing maintenance will be fairly simple for upstream, and changes only really need to consider exception and string/byte literal syntax, for the most part.
  2. Merge this new simplejson with stdlib json for 3.3.

I looked at step 1 a few weeks ago and have made some progress with it. I've just forked simplejson on Github and posted my changes to my fork:

https://github.com/vsajip/simplejson

All 136 tests pass on Python 2.7 (just as a control/sanity check), and on Python 3.2, there are 4 failures and 12 errors - see complete results at

https://gist.github.com/923019

I haven't looked at the C extension yet, just the Python code. I believe most of the test failures will be down to string literals in the tests which should be bytes, e.g. test_unicode.py:test_ensure_ascii_false_bytestring_encoding.

So, it looks quite encouraging, and if you think my approach has merit, please take a look at my fork, and give feedback/join in!

Note that I used the same approach when porting pip/virtualenv to Python 3, which seems to have gone quite smoothly :-)

Regards,

Vinay Sajip

More information about the Python-Dev mailing list