[Python-Dev] timeit module (original) (raw)

Connelly Barnes connellybarnes at yahoo.com
Tue Jan 17 23:08:36 CET 2006

Connelly Barnes wrote:

Hi,

Perhaps I am the only one bothered by the timeit module, but it seems poorly designed to me. First of all, it should use a geometric series with a timeout value to detect how many iterations it should perform. Currently, the user is required to manually specify the number of iterations (the default is 1 The provision of a default value generally seems to chop the legs from

your argument that the number of iterations is required.

Suppose we wish to time two functions: f1 and f2. Function f1 takes 1e-5 seconds to run, and function f2 takes 0.1 seconds to run. If these functions are defined in interactive mode, we can time them with timeit via:

timeit.Timer('f1()', 'from main import f1').timeit(1000000)/1000000 timeit.Timer('f2()', 'from main import f2').timeit(10)/10

The first line gives the approximate time to run f1 and the second line gives the approximate time to run f2. The number parameter to timeit() must be supplied for the second line, since the default 1 million iterations would take too long to be practical. The number parameter to timeit() is also supplied for the first line, since it makes the code more robust in case the default argument value in timeit() is changed.

My first observation is that this is a lot of contortion to get the time of functions f1() and f2(). Something simpler would be nice. For example, using Recipe 440657 this can be simplified to:

pytime.pytime(f1, ()) pytime.pytime(f2, ())

Here the timing routine calculates the number of iterations to make, and the timing routine divides by the number of iterations of the supplied callable.

This is also convenient if you move from a computer to one that is faster or slower. If you move to a computer that is 10 times slower, then the timeit.Timer() code above will take 10 times as long to run. The usual solution is for the programmer to adjust the number of iterations as he changes computers (so he doesn't wait around all day for his benchmarks to run). Likewise, if one optimizes f2() so that it runs 100 times faster, one would again have to adjust the number of iterations in order to gain an accurate timing result (if the total time spent in the timing routine is less than 1 msec, then the resulting time for f2() is off by more than 10% on my Windows machine, which is undesirable, so the number of iterations cannot be made too small).

Summary: Module timeit is complicated to use on an existing function, it does not time the number of seconds to execute 1 iteration of the function (this is the value one usually wants in benchmarking, or its reciprocal), and it requires that the end user tweak the number of iterations as he or she changes between computers with different CPU speed, or as he or she optimizes code. It is easy to make a cleaner module that fixes these problems. It might be good to incorporate such a module into Python, due to its general usefulness.

million). If the user optimizes his or her code, then the number of iterations must be changed. If the user moves to a slower or faster computer, then the number of iterations must be changed again. This is annoying.

What? The purpose of timeit is to give an approximation to the length of time to run a specific piece ofcode. Why must the number of iterations be changed when moving to a slower or faster computer? Secondly, there should be a way to time a callable directly. That is, without finding the string name of the callable and using a string "import X" statement. These contortions violate rules #1 and #3 of the Zen of Python. Presumably for benchmarking purposes the function call overhead would be present for all compaered techniques. Do you mean rules #0 and #2?

I mean "Beautiful is better than ugly" and "Simple is better than complex."

> [...] regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/

Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

More information about the Python-Dev mailing list