[Python-Dev] file.readinto performance regression in Python 3.2 vs. 2.7? (original) (raw)

Eli Bendersky eliben at gmail.com
Fri Nov 25 07:38:48 CET 2011

On Thu, 24 Nov 2011 20:53:30 +0200 Eli Bendersky <eliben at gmail.com> wrote: > > Sure. Updated the default branch just now and built: > > $1 -m timeit -s'import filereadbytearray' 'filereadbytearray.justread()' > 1000 loops, best of 3: 1.14 msec per loop > $1 -m timeit -s'import filereadbytearray' > 'filereadbytearray.readandcopy()' > 100 loops, best of 3: 2.78 msec per loop > $1 -m timeit -s'import filereadbytearray' 'filereadbytearray.readinto()' > 1000 loops, best of 3: 1.6 msec per loop > > Strange. Although here, like in python 2, the performance of readinto is > close to justread and much faster than readandcopy, but justread itself is > much slower than in 2.7 and 3.2!

This seems to be a side-effect of http://hg.python.org/cpython/rev/f8a697bc3ca8/ Now I'm not sure if these numbers matter a lot. 1.6ms for a 3.6MB file is still more than 2 GB/s.

Just to be clear, there were two separate issues raised here. One is the speed regression of readinto() from 2.7 to 3.2, and the other is the relative slowness of justread() in 3.3

Regarding the second, I'm not sure it's an issue because I tried a larger file (100MB and then also 300MB) and the speed of 3.3 is now on par with 3.2 and 2.7

However, the original question remains - on the 100MB file also, although in 2.7 readinto is 35% faster than readandcopy(), on 3.2 it's about the same speed (even a few % slower). That said, I now observe with Python 3.3 the same speed as with 2.7, including the readinto() speedup - so it appears that the readinto() regression has been solved in 3.3? Any clue about where it happened (i.e. which bug/changeset)?

Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20111125/83d16831/attachment.html>

More information about the Python-Dev mailing list