[Python-Dev] pystone rant [was] proposal+patch: sys.gettickeraccumulation() (original) (raw)

Raymond Hettinger python at rcn.com
Mon Dec 6 01:33:17 CET 2004

(Old message)

> =) Parrotbench and PyBench would be enough for me to sign off on any > performance hit. There is also pystone.

pystone is good a predicting the relative performance of python apps on difference hardware/software environments.

As a tool for evaluating proposed language changes, it completely sucks and should never be used for anything other than an extra informational datapoint.

The foremost issue is that it exercises only a tiny portion of the language.

Its design makes it worse than totally useless for evaluating eval-loop overhead. It runs a loop twice, once with benchmarking code and once without. Only the difference is reported. In theory, that means that all eval-loop speedups and slowdowns are netted out of the result. In practice, the reported result may indicate exactly the opposite of reality because the empty loop has vastly different cache effects than the benchmarked loop.

For useful timings, run timeit on the specific feature in question. Then check for overall impact using pybench, parrotbench, and test_decimal.

Raymond

More information about the Python-Dev mailing list