[Python-Dev] Re: Are we collecting benchmark results across machines (original) (raw)

Guido van Rossum guido at python.org
Fri Jan 2 13:23:20 EST 2004

On this box (3.05G P4, 512M, Fedora Core 1):

Using Python2.3.3: real 0m16.027s user 0m14.940s sys 0m0.280s Pystone(1.1) time for 50000 passes = 1.43 This machine benchmarks at 34965 pystones/second

Using current-CVS python (and the bytecode compiled with 2.3.3): real 0m14.556s user 0m14.130s sys 0m0.280s Pystone(1.1) time for 50000 passes = 1.41 This machine benchmarks at 35461 pystones/second (Hm: should the default number of passes for pystone be raised? One and a half seconds seems rather a short time for a benchmark...)

Probably. Maybe you can try the new pystone from CVS which has a command line option. (If I were to write it over I'd use the strategy from timeit, which has a self-adjusting strategy to pick an appropriate number of loops.)

I multiplied the two numbers (pystones/sec and parrotbench seconds) for your two runs and found the product to be much higher for your first run than for the second. This is suspicious (perhaps points at too much variation in the pystone timing); for contrast, Skip's two runs before and after rebooting give a product within 0.05 percent of each other).

Hm, of course this could also have to do with Python versions. Let me try...

Yes, it looks like for me, on one machine I have handy here, Python 2.3.3 gives a higher product than current CVS (though not by as much as you measured).

So what does this product mean? It's higher if pystone is faster, or parrotbench is slower. Any factor that makes both faster (or slower) by the same amount has no effect. Ah (thinking aloud here) the unit is "pystones". Not pystones per second, just pystones. And this leads to an interpretation: it is how many pystone loops execute in the time needed to complete the parrotbench benchmark (so the unit is really "pystones per parrotbench").

So what makes a machine run more pystones in a parrotbench?

I don't know enough about CPUs and caches to draw a conclusion, and I've got to start doing real work today...

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list