Issue 7451: improve json decoding performance (original) (raw)

This issue has been migrated to GitHub: https://github.com/python/cpython/issues/51700

classification

Title: improve json decoding performance
Type: performance Stage: resolved
Components: Library (Lib) Versions: Python 3.2

process

Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, bob.ippolito, pitrou, rhettinger, vkuznet
Priority: normal Keywords: patch

Created on 2009-12-07 23:08 by pitrou, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
json-opts2.patch pitrou,2009-12-07 23:08
json-opts3.patch pitrou,2009-12-08 00:54
json-opts4.patch pitrou,2010-09-04 20:12
Messages (11)
msg96082 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-07 23:08
This patch does two things: - it speeds up decoding of JSON objects by doing fewer temporary allocations - it reduces memory use of decoded JSON objects by reusing key strings when they appear several times in a JSON string
msg96093 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-12-08 00:08
The problem with this is that it makes json in Python diverge even more from the simplejson trunk.
msg96097 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-08 00:42
Unless Bob applies it to simplejson as well, that is.
msg96098 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-08 00:53
I've just noticed that there is a persistent decoder instance in `json._default_decoder`. To avoid keeping references to past keys forever, this updated patch clears the memo dict when a string has finished decoding.
msg96099 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-08 00:54
Here is the patch.
msg96169 - (view) Author: Valentin Kuznetsov (vkuznet) Date: 2009-12-09 15:23
I wonder if you can make a patch for 2.6 python branch.
msg96171 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-09 15:58
> I wonder if you can make a patch for 2.6 python branch. No, 2.6 doesn't have the same C accelerator in the first place.
msg96903 - (view) Author: Bob Ippolito (bob.ippolito) * (Python committer) Date: 2009-12-27 11:10
I applied most of this patch to r206 of simplejson trunk
msg96908 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-27 12:02
Le dimanche 27 décembre 2009 à 11:10 +0000, Bob Ippolito a écrit : > Bob Ippolito <bob@redivi.com> added the comment: > > I applied most of this patch to r206 of simplejson trunk Thank you. Will you port it to CPython yourself or would you prefer someone else to do it?
msg115611 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-04 20:12
Updated patch against py3k.
msg115612 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-04 20:17
Committed in r84505.
History
Date User Action Args
2022-04-11 14:56:55 admin set github: 51700
2010-09-04 20:17:11 pitrou set status: open -> closedresolution: fixedmessages: + stage: commit review -> resolved
2010-09-04 20:12:45 pitrou set files: + json-opts4.patchstage: patch review -> commit reviewmessages: + versions: - Python 2.7
2009-12-27 12:02:23 pitrou set messages: +
2009-12-27 11:10:33 bob.ippolito set messages: +
2009-12-09 15:58:56 pitrou set messages: +
2009-12-09 15:23:23 vkuznet set messages: +
2009-12-08 00:54:35 pitrou set files: + json-opts3.patchmessages: +
2009-12-08 00:53:32 pitrou set messages: +
2009-12-08 00:42:47 pitrou set messages: +
2009-12-08 00:08:24 benjamin.peterson set nosy: + benjamin.petersonmessages: +
2009-12-07 23:08:19 pitrou create