[Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()? (original) (raw)

M.-A. Lemburg mal at egenix.com
Tue Apr 3 14:26:25 CEST 2012

Victor Stinner wrote:

You seem to have missed the episode where I explained that caching the last value in order to avoid going backwards doesn't work -- at least not if the cached value is internal to the API implementation.

Yes, and I can't find it by briefly searching my mail. I haven't had the energy to follow every bit of this discussion because it has become completely insane. I'm trying to complete the PEP, but I didn't add this part yet. Of course we cannot promise not moving backwards, since there is a 64 bit wraparound some years in the future. Some years? I computed 584.5 years, so it should not occur in practice. 32-bit wraparound is a common issue which occurs in practice on Windows (49.7 days wraparound), and I propose a workaround in the PEP (already implemented in the related issue). Here's actual code from production: BOOL WINAPI QueryPerformanceCounterCCP( LARGEINTEGER* li ) { static LARGEINTEGER last = {0}; BOOL ok = QueryPerformanceCounter(li); if( !ok ) { return FALSE; } Did you already see it failing in practice? Python ignores the return value and only uses the counter value. Even negative delta values of time are usually harmless on the application level. A curiosity, but harmless. It depends on your usecase. For a scheduler or to implement a timeout, it does matter. For a benchmark, it's not an issue because you usually repeat a test at least 3 times. Most advanced benchmarked tools gives a confidence factor to check if the benchmark ran fine or not. I am offering empirical evidence here from hundreds of thousands of computers over six years: For timing and benchmarking, QPC is good enough, and will only be as precise as the hardware and operating system permits, which in practice is good enough. The PEP contains also different proofs that QPC is not steady, especially on virtual machines.

I'm not sure I understand what you are after here, Victor. For benchmarks it really doesn't matter if one or two runs fail due to the timer having a problem: you just repeat the run and ignore the false results (you have such issues in all empirical studies). You're making things needlessly complicated here.

Regarding the approach to try to cover all timing requirements into a single time.steady() API, I'm not convinced that this is good approach. Different applications have different needs, so it's better to provide interfaces to what the OS has to offer and let the application decide what's best.

If an application wants to have a monotonic clock, it should use time.monotonic(). The OS doesn't provide it, you get an AttributeError and revert to some other function, depending on your needs.

Having a time.steady() API make this decision for you, is not going to make your application more portable, since the choice will inevitably be wrong in some cases (e.g. going from CLOCK_MONOTONIC to time.time() as fallback).

BTW: You might also want to take a look at the systimes.py module in pybench. We've been through discussions related to benchmark timing in 2006 already and that module summarizes the best practice outcome :-)

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Apr 03 2012)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

2012-04-03: Python Meeting Duesseldorf today

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

More information about the Python-Dev mailing list