[Python-Dev] Store timestamps as decimal.Decimal objects (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Wed Feb 1 11:43:24 CET 2012

On Wed, Feb 1, 2012 at 6:03 PM, Victor Stinner <victor.stinner at haypocalc.com> wrote:

2012/2/1 Nick Coghlan <ncoghlan at gmail.com>:

The secret to future-proofing such an API while only using integers lies in making the decimal exponent part of the conversion function signature:

 def fromcomponents(integer, fraction=0, exponent=-9):  return Decimal(integer) + Decimal(fraction) * Decimal((0, (1,), exponent)) The fractional part is not necessary related to a power of 10. An earlier version of my patch used also powers of 10, but it didn't work (loose precision) for QueryPerformanceCounter() and was more complex than the new version. NTP timestamp uses a fraction of 2**32. QueryPerformanceCounter() (used by time.clock() on Windows) uses the CPU frequency.

If a callback protocol is used at all, there's no reason those details need to be exposed to the callbacks. Just choose an appropriate exponent based on the precision of the underlying API call.

We may need more information when adding a new timestamp formats later. If we expose the "internal structure" used to compute any timestamp format, we cannot change the internal structure later without breaking (one more time) the API.

You're assuming we're ever going to want timestamps that are something more than just a number. That's a huge leap (much bigger than increasing the precision, which is the problem we're dealing with now).

With arbitrary length integers available, "integer, fraction, exponent" lets you express numbers to whatever precision you like, just as decimal.Decimal does (more on that below).

My patch is similar to your idea except that everything is done internally to not have to expose internal structures, and it doesn't touch decimal or datetime modules. It would be surprising to add a method related to timestamp to the Decimal class.

No, you wouldn't add a timestamp specific method to the Decimal class

This strategy would have negligible performance impact There is no such performance issue: time.time() performance is exactly the same using my patch. Depending on the requested format, the performance may be better or worse. But even for Decimal, I think that the creation of Decimal is really "fast" (I should provide numbers :-)).

But this gets us to my final question. Given that Decimal supports arbitrary precision, why increase the complexity of the underlying API by supporting other output types? If you're not going to support arbitrary callbacks, why not just have a "high precision" flag to request Decimal instances and be done with it? datetime, timedelta and so forth would be able to get everything they needed from the Decimal value.

As I said in my last message, both a 3-tuple (integer, fraction, exponent) based callback protocol effectively supporting arbitrary output types and a boolean flag to request Decimal values make sense to me and I could argue in favour of either of them. However, I don't understand the value you see in this odd middle ground of "instead of picking 1 arbitrary precision timestamp representation, whether an integer triple or decimal.Decimal, we're going to offer a few different ones and make you decide which one of them you actually want every time you call the API". That's seriously ducking our responsibilities as language developers - it's our job to make that call, not each user's.

Given the way the discussion has gone, my preference is actually shifting strongly towards just returning decimal.Decimal instances when high precision timestamps are requested via a boolean flag. The flag isn't pretty, but it works, and the extra flexibility of a "type" parameter or a callback protocol doesn't really buy us anything once we have an output type that supports arbitrary precision.

FWIW, I did a quick survey of what other languages seem to offer in terms of high resolution time interfaces:

However, I don't know enough about how the APIs in those languages work to do sensible searches. It doesn't appear to be a cleanly solved problem anywhere, though.

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list