[Python-Dev] Pure pickle bechmark. (original) (raw)

INADA Naoki songofacandy at gmail.com
Sun Jul 9 19:17:30 EDT 2017

I don't know this is relating to your case.

When I saw Victor's report [1], I researched why Python 3 is slower than Python 2 on unpickle_pure_python benchmark.

[1] https://mail.python.org/pipermail/speed/2017-February/000503.html

And I found Python 2 and 3 uses different version of pickle format.

Current Python 3 uses "framing" format. While unpickling, read(1) is very performance critical. Python 2 uses cStringIO.read which is implemented in C. On the other hand, Python 3 uses _Unframer.read which is implemented in Python.

Since this is not relating to "first import time", I don't know this is what you want to optimize. (Since _pickle is used for normal case, pure Python unpickle performance is not a common problem).

If you want to optimize it, _Unframer uses BytesIO internally and performance critical part may be able to call BytesIO.read directly instead of _Unframer.read.

Regards, INADA Naoki <songofacandy at gmail.com>

On Sun, Jul 9, 2017 at 11:08 PM, Bhavishya <bhavishyagopesh at gmail.com> wrote:

Hello,

1).I was going through the code of python pickle to search any optimization possibility.But the only thing that I found very alarming was again the import time(I tried with lazy-import but it didn't helped much.) I found py3 to be ~45 times slower on initial imports(very raw measure..using "time." ) as compared to py2 on an usual example. py3-> ./python -c ' favoritecolor = { "lion": "yellow", "kitty": "red" } pickle.dump( favoritecolor, open( "save.p", "wb" ) )' 0.009715557098388672(time taken to do initial imports...measured using time.time() ) py2-> ./python -c ' favoritecolor = { "lion": "yellow", "kitty": "red" } pickle.dump( favoritecolor, open( "save.p", "wb" ) )' 0.000236034393311(time taken to do initial imports...measured using time.time() ) Do you have any thought/ideas on improving this?

Thank You.

More information about the Python-Dev mailing list