[Python-Dev] Re: [Python-checkins] python/dist/src/Lib timeit.py, 1.15, 1.16 (original) (raw)

Guido van Rossum guido at python.org
Sun Jan 4 11:41:07 EST 2004

> > Make timings more consistent by temporarily disabling GC. > > Shouldn't this be an option at best?

I think not. The default behavior should be the one that gives the most consistent timings between runs.

If the timed code's malloc behavior is predictable (and I say it usually is!) then the GC behavior should (in theory) also be predictable, at least if you do a full GC before starting.

But I haven't experimented with this, and from your reversal I take it that you have.

> What if I want to time some code > that expects GC to be on?

>>> Timer(stmt, 'import gc; gc.enable()'.timeit() If desired, I can add a note to that effect in the docs.

Please do.

However, GC is such a random event (in terms of when it occurs, what it affects, how much time it takes, and the state of the system) that you would always want it off unless you're specifically measuring GC performance across two sets of code that handle GC differently.

If I have a piece of code that allocates so much as to trigger GC regularly, I'd want the GC behavior included in the timings.

GC performance does make a difference when measuring the performance of a whole, memory intensive application; however, that tends to fall outside the scope and purpose of the module.

Usually. That's why I'm asking for a way to change the default behavior.

If you really want an option, I can put it in, but why clutter the API with something that few people care about or understand.

Your setup suggestion is fine.

An alternative is to stick with my original idea of running a gc.collect() before each timing pass. I would have done that but Tim's comment left me with doubts about whether it was the right thing to do. On the surface it makes sense and empirical tests show that the timings do become more stable. However, trashing the cache before each run may create timings that don't reflect a normal operating environment (though the timeit environment is already anything but normal).

Here I agree with Tim.

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list