[Python-Dev] [GSoC] Developing a benchmark suite (for Python 3.x) (original) (raw)

Stefan Behnel stefan_ml at behnel.de
Fri Apr 8 12🔞51 CEST 2011

Maciej Fijalkowski, 08.04.2011 11:41:

On Fri, Apr 8, 2011 at 11:22 AM, Stefan Behnel<stefanml at behnel.de> wrote:

Jesse Noller, 07.04.2011 22:28:

On Thu, Apr 7, 2011 at 3:54 PM, Anthony Scopatz wrote:

Hi Daniel, Thanks for putting this together. I am a huge supporter of benchmarking efforts. My brief comment is below. On Wed, Apr 6, 2011 at 11:52 AM, DasIch wrote:

1. Definition of the benchmark suite. This will entail contacting developers of Python implementations (CPython, PyPy, IronPython and Jython), via discussion on the appropriate mailing lists. This might be achievable as part of this proposal.

If you are reaching out to other projects at this stage, I think you should also be in touch with the Cython people (even if its 'implementation' sits on top of CPython). As a scientist/engineer what I care about is how Cython benchmarks to CPython. I believe that they have some ideas on benchmarking and have also explored this space. Their inclusion would be helpful to me thinking this GSoC successful at the end of the day (summer). Thanks for your consideration. Be Well Anthony Right now, we are talking about building "speed.python.org" to test the speed of python interpreters, over time, and alongside one another - cython is not an interpreter. Would you also want to exclude Psyco then? It clearly does not qualify as a Python interpreter. Why not? it does run those benchmarks just fine.

Sure.

Cython is out of scope for this.

Why? It should be easy to integrate Cython using pyximport. Basically, all you have to do is register the pyximport module as an import hook. Cython will then try to compile the imported Python modules and fall back to the normal .py file import if the compilation fails for some reason. then it's fine to include it. we can even include it now in speed.pypy.org that way. would it compile django?

Never tried. Likely not completely, but surely some major parts of it. That's the beauty of it - it just falls back to CPython. :) If we're lucky, it will manage to compile some performance critical parts without modifications. In any case, it'll be trying to compile each module.

So, once CPython is up and running in the benchmark test, adding Cython should be as easy as copying the configuration, installing Cython and adding two lines to site.py. can you provide a simple command line tool for that? I want essentially to run ./cython-importing-stuff some-file.py

You can try

 python -c 'import pyximport; \
            pyximport.install(pyimport=True); \
            exec("somefile.py")'

You may want to configure the output directory for the binary modules, though, see

https://github.com/cython/cython/blob/master/pyximport/pyximport.py#L343

Please also take care to provide suitable gcc CFLAGS, e.g. "-O3 -march=native" etc.

Obviously, we'd have to integrate a build of the latest Cython development sources as well, but it's not like installing a distutils enabled Python package from sources is so hard that it pushes Cython out of scope for this GSoC. no, that's fine. My main concern is - will cython run those benchmarks?

In the worst case, they will run at CPython speed with uncompiled modules.

and will you complain if we don't provide a custom cython hacks? (like providing extra type information)

I don't consider providing extra type information a hack. Remember that they are only used for additional speed-ups in cases where the author is smarter than the compiler. It will work just fine without them.

Stefan

More information about the Python-Dev mailing list