[Python-Dev] memcmp performance (original) (raw)

Richard Saunders richismyname at me.com
Thu Oct 20 23:23:30 CEST 2011

Hey,

I have been doing some performance experiments with memcmp, and I was surprised that memcmp wasn't faster than it was in Python.  I did a whole,  long analysis and came up with some very simple results.

Paul Svensson suggested I post as much as I can as text, as people would be more likely to read it. So, here's the basic ideas:

(1) memcmp is surprisingly slow on some Intel gcc platforms (Linux)         On several Linux, Intel platforms, memcmp was 2-3x slower than          a simple, portable C function (with some optimizations)

(2) The problem: If you compile C programs with gcc with any optimization on,       it will replace all memcmp calls with an assembly language stub: rep cmpsb      instead of the memcmp call.

(3) rep cmpsb seems like it would be faster, but it really isn't:        this completely bypasses the memcmp.S, memcmp_sse3.S       and memcmp_sse4.S in glibc which are typically  faster.

(4) The basic conclusion is that the Python baseline on       Intel gcc platforms should probably be compiled with -fno-builtin-memcmp      so we "avoid" gcc's  memcmp optimization.

The numbers are all in the paper: I will endeavor to try to generate a text form of all the tables so it's easier to read.  This is much first in the Python dev arena, so I went a little overboard with my paper below. ;)

  Gooday,

  Richie

Before I put in a tracker bug report, I wanted to present my findings and make sure they were repeatable to others (isn't that the nature of science? ;)   as well as offer discussion.

The analysis is a pdf and is here:  http://www.picklingtools.com/study.pdf The testcases are a tarball here:  http://www.picklingtools.com/PickTest5.tar.gz I have three basic recommendations in the study: I am curious what other people think.

-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20111020/c910868a/attachment.html>

More information about the Python-Dev mailing list