[Python-Dev] Benchmarks: Comparison between Python 2.7 and Python 3.6 performance (original) (raw)

Victor Stinner victor.stinner at gmail.com
Fri Nov 4 08:28:36 EDT 2016

Hi,

You may know that I'm working on benchmarks. I regenerated all benchmark results of speed.python.org using performance 0.3.2 (benchmark suite). I started to analyze results.

All results are available online on the website:

https://speed.python.org/

To communicate on my work on benchmarks, I tweeted two pictures:

"sympy benchmarks: Python 3.6 is between 8% and 48% faster than Python 2.7 #python #benchmark": https://twitter.com/VictorStinner/status/794289596683210760

"Python 3.6 is between 25% and 54% slower than Python 2.7 in the following benchmarks": https://twitter.com/VictorStinner/status/794305065708376069

Many people were disappointed that Python 3.6 can be up to 54% slower than Python 2.7. In fact, I know many reasons which explain that, but it's hard to summarize them in 140 characters ;-)

For example, Python 3.6 is 54% slower than Python 2.7 on the benchmark pycrypto_aes. This benchmark tests a pure Python implementation of the crypto cipher AES. You may know that CPython is slow for CPU intensive functions, especially on integer and floatting point numbers.

"int" in Python 3 is now "long integers" by default, which is known to be a little bit slower than "short int" of Python 2. On a more realistic benchmark (see other benchmarks), the overhead of Python 3 "long int" is negligible.

AES is a typical example stressing integers. For me, it's a dummy benchmark: it doesn't make sense to use Python for AES: modern CPUs have an hardware implemention which is super fast.

Well, I didn't have time to analyze in depth individual benchmarks. If you want to help me, here is the source code of benchmarks: https://github.com/python/performance/blob/master/performance/benchmarks/

Raw results of Python 3.6 compared to Python 2.7:

$ python3 -m perf compare_to 2016-11-03_15-36-2.7-91f024fc9b3a.json.gz 2016-11-03_15-38-3.6-c4319c0d0131.json.gz -G --min-speed=5 Slower (40):

Faster (15):

Benchmark hidden because not significant (8): 2to3, dulwich_log, nbody, pidigits, regex_dna, tornado_http, unpack_sequence, unpickle Ignored benchmarks (3) of 2016-11-03_15-36-2.7-91f024fc9b3a.json: hg_startup, pyflate, spambayes

Please ignore call_method, call_method_slots, call_method_unknown benchmarks: it seems like I had an issue on the benchmark server. I was unable to reproduce he 70% slowdown on my laptop.

I attached the two compressed JSON files to this email if you want to analyze them yourself.

I hope that my work on benchmarks will motive some developers to look closer at Python 3 performance to find interesting optimizations ;-)

Victor -------------- next part -------------- A non-text attachment was scrubbed... Name: 2016-11-03_15-36-2.7-91f024fc9b3a.json.gz Type: application/x-gzip Size: 107594 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-dev/attachments/20161104/20759909/attachment-0002.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: 2016-11-03_15-38-3.6-c4319c0d0131.json.gz Type: application/x-gzip Size: 102546 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-dev/attachments/20161104/20759909/attachment-0003.bin>

More information about the Python-Dev mailing list