[Python-Dev] Python parser performance optimizations (original) (raw)

Guido van Rossum guido at python.org
Fri Sep 16 10:44:44 EDT 2016

OK, but if nobody responds within a week we should close it. IMO there's no value in keeping things around that nobody is going to apply. I don't expect that a year from now we'll suddenly a surge of interest in this patch, sorry.

On Fri, Sep 16, 2016 at 4:25 AM, Artyom Skrobov <Artyom.Skrobov at arm.com> wrote:

Thank you very much for your comments,

I appreciate that we're all volunteers, and that if nobody fancies reviewing a big invasive patch, then it won't get reviewed. Still, I want to note that the suggested optimization has a noticeable positive effect on many benchmarks -- even though the effect may only become of practical value in such uncommon use cases as parsing huge data tables. As I found out later, JSON wasn't a viable option for storing dozens of megabytes of deeply-nested data, either. To get acceptable deserialization performance, I eventually had to resort to pickled files.

-----Original Message----- From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum Sent: 15 September 2016 17:01 To: Artyom Skrobov Cc: python-dev at python.org; brett at python.org; jimjjewett at gmail.com; nd Subject: Re: [Python-Dev] Python parser performance optimizations I wonder if this patch could just be rejected instead of lingering forever? It clearly has no champion among the current core devs and therefore it won't be included in Python 3.6 (we're all volunteers so that's how it goes). The use case for the patch is also debatable: Python's parser wasn't designed to efficiently parse huge data tables like that, and if you have that much data, using JSON is the right answer. So this doesn't really scratch anyone's itch except of the patch author (Artyom). From a quick look it seems the patch is very disruptive in terms of what it changes, so it's not easy to review. I recommend giving up, closing the issue as "won't fix", recommending to use JSON, and moving on. Sometimes a change is just not worth the effort. --Guido On Tue, Aug 9, 2016 at 1:59 AM, Artyom Skrobov <Artyom.Skrobov at arm.com> wrote: > Hello, > > > > This is a monthly ping to get a review on http://bugs.python.org/ issue26415 > -- “Excessive peak memory consumption by the Python parser”. > > > > Following the comments from July, the patches now include updating Misc/NEWS > and compiler.rst to describe the change. > > > > The code change itself is still the same as a month ago. > > > > > > From: Artyom Skrobov > Sent: 07 July 2016 15:44 > To: python-dev at python.org; steve at pearwood.info; mafagafogigante at gmail.com; > greg.ewing at canterbury.ac.nz > Cc: nd > Subject: RE: Python parser performance optimizations > > > > Hello, > > > > This is a monthly ping to get a review on http://bugs.python.org/ issue26415 > -- “Excessive peak memory consumption by the Python parser”. > > The first patch of the series (an NFC refactoring) was successfully > committed earlier in June, so the next step is to get the second patch, “the > payload”, reviewed and committed. > > > > To address the concerns raised by the commenters back in May: the patch > doesn’t lead to negative memory consumption, of course. The base for > calculating percentages is the smaller number of the two; this is the same > style of reporting that perf.py uses. In other words, “200% less memory > usage” is a threefold shrink. > > > > The absolute values, and the way they were produced, are all reported under > the ticket. > > > > > > From: Artyom Skrobov > Sent: 26 May 2016 11:19 > To: 'python-dev at python.org' > Subject: Python parser performance optimizations > > > > Hello, > > > > Back in March, I’ve posted a patch at http://bugs.python.org/issue26526 -- > “In parsermodule.c, replace over 2KLOC of hand-crafted validation code, with > a DFA”. > > > > The motivation for this patch was to enable a memory footprint optimization, > discussed at http://bugs.python.org/issue26415 > > My proposed optimization reduces the memory footprint by up to 30% on the > standard benchmarks, and by 200% on a degenerate case which sparked the > discussion. > > The run time stays unaffected by this optimization. > > > > Python Developer’s Guide says: “If you don’t get a response within a few > days after pinging the issue, then you can try emailing > python-dev at python.org asking for someone to review your patch.” > > > > So, here I am. > > _> ________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org >

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20160916/e998f5d3/attachment-0001.html>

More information about the Python-Dev mailing list