[Python-Dev] PEP 418: Add monotonic clock (original) (raw)

Guido van Rossum guido at python.org
Wed Mar 28 17:47:31 CEST 2012

On Wed, Mar 28, 2012 at 8:08 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

On Thu, Mar 29, 2012 at 12:42 AM, Guido van Rossum <guido at python.org> wrote:

As I said, I think the caching idea is bad. We may have to settle for semantics that are less than perfect -- presumably if you are doing benchmarking you just have to throw away a bad result that happened to be affected by a clock anomaly, and if you are using timeouts, retries are already part of life. I agree caching doesn't solve the problems that are solved by an OS level monotonic clock, but falling back to an unmodifided time.time() result instead doesn't solve those problems either. Falling back to time.time() just gives you the status quo: time may jump forwards or backwards by an arbitrary amount between any two calls. Cached monotonicity just changes the anomalous modes to be time jumping forwards, or time standing still for an extended period of time. The only thing the caching provides is that it becomes a reasonable fallback for a function called time.monotonic() - it is a monotonic clock that meets the formal contract of the function, it's just nowhere near as good or effective as one the OS can provide.

TBH, I don't think like this focus on monotonicity as the most important feature.

Forward jumping anomalies aren't as harmful, are very hard to detect in the first place and behave the same regardless of the presence of caching, so the interesting case to look at is the difference in failure modes when the system clock jumps backwards.

Agreed.

For benchmarking, a caching clock will produce a zero result instead of a negative result. Zeros aren't quite as obviously broken as negative numbers when benchmarking, but they're still sufficiently suspicious that most benchmarking activities will flag them as anomalous. If the jump back was sufficiently small that the subsequent call still produces a higher value than the original call, then behaviour reverts to being identical.

So for benchmarking we don't care about jumps, really, and the caching version is slightly less useful.

For timeouts, setting the clock back means your operation will take longer to time out than you expected. This problem will occur regardless of whether you were using cached monotonicity (such that time stands still) or the system clock (such that time actually goes backwards). In either case, your deadline will never be reached until the backwards jump has been cancelled out by the subsequent passage of time.

Where in the stdlib do we actually calculate timeouts instead of using the timeouts built into the OS (e.g. select())?

I think it would be nice if we could somehow use the same clock as the OS uses to implement timeouts.

I want the standard library to be able to replace its time.time() calls with time.monotonic().

Where in the stdlib? (I'm aware of threading.py. Any other places?)

The only way we can do that without breaking cross-platform compatibility is if time.monotonic() is guaranteed to exist, even when the platform only provides time.time(). A dumb caching fallback implementation based on time.time() is the easiest way to achieve that withou making a complete mockery of the "monotonic()" name.

Yeah, so maybe it's a bad name. :-)

There is then a different use case, which is 3.3+ only code which wants to fail noisily when there's no OS level monotonic support - the application developer really does want to fail immediately if there's no OS level monotonic clock available, instead of crossing your fingers and hoping you don't hit a clock adjustment glitch (crossing your fingers has, I'll point out, been the only option for all previous versions of Python, so it clearly can't be that scary a prospect).

So, rather than making time.monotonic() something that the *standard library can't use*, I'd prefer to address that second use case by exposing the OS level monotonic clock as time.osmonotonic() only when it's available. That way, the natural transition for old time.time() based code is to time.monotonic() (with no cross-platform support implications), but time.osmonotonic() also becomes available for the stricter use cases.

I'd be happier if the fallback function didn't try to guarantee things the underlying clock can't guarantee. I.e. I like the idea of having a function that uses some accurate OS clock if one exists but falls back to time.time() if not; I don't like the idea of that new function trying to interpret the value of time.time() in any way. Applications that need the OS clock's guarantees can call it directly. We could also offer something where you can introspect the properties of the clock (or clocks) so that an app can choose the best clock depending on its needs.

To summarize my problem with the caching idea: take a simple timeout loop such as found in several places in threading.py.

def wait_for(delta, eps):

Wait for delta seconds, sleeping eps seconds at a time

deadline = now() + delta while now() < deadline: sleep(eps)

If the now() clock jumps backward after the initial call, we end up waiting too long -- until either the clock jumps forward again or until we've made up the difference. If the now() clock jumps forward after the initial call, we end up waiting less time, which is probably not such a big problem (though it might).

But now consider a caching clock, and consider that the system clock made a jump backwards before this function is called. The cache prevents us from seeing it, so the initial call to now() returns the highest clock value seen so far. And until the system clock has caught up with that, now() will return the same value over and over -- so WE STILL WAIT TOO LONG.

My conclusion: you can't win this game by forcing the clock to return a monotonic value. A better approach might be to compute how many sleep(eps) calls we're expected to make, and to limit the loop to that -- although sleep() doesn't make any guarantees either about sleeping too short or too long. Basically, if you do sleep(1) and find that your clock didn't move (enough), you can't tell the difference between a short sleep and a clock that jumped back. And if your clock moved to much, you still don't know if the problem was with sleep() or with your clock.

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list