[Python-Dev] Stop using timeit, use perf.timeit! (original) (raw)

Steven D'Aprano steve at pearwood.info
Fri Jun 10 21:45:49 EDT 2016

On Fri, Jun 10, 2016 at 11:22:42PM +0200, Victor Stinner wrote:

2016-06-10 20:47 GMT+02:00 Meador Inge <meadori at gmail.com>: > Apologies in advance if this is answered in one of the links you posted, but > out of curiosity was geometric mean considered? > > In the compiler world this is a very common way of aggregating performance > results.

FYI I chose to store all timings in the JSON file. So later, you are free to recompute the average differently, compute other statistics, etc. I saw that the CPython benchmark suite has an option to compute the geometric mean. I don't understand well the difference with the arithmeric mean. Is the geometric mean recommended to aggregate results of different (unrelated) benchmarks, or also even for multuple runs of a single benchmark?

The Wikipedia article discusses this, but sits on the fence and can't decide whether using the gmean for performance results is a good or bad idea:

https://en.wikipedia.org/wiki/Geometric_mean#Properties

Geometric mean is usually used in finance for averaging rates of growth:

https://www.math.toronto.edu/mathnet/questionCorner/geomean.html

If you express your performances as speeds (as "calculations per second") then the harmonic mean is the right way to average them.

-- Steve

More information about the Python-Dev mailing list