Fix inconsistent rounding of 0.5 when formatted to 0 decimal places by ajtribick · Pull Request #102935 · rust-lang/rust (original) (raw)

I'm sorry this is going out of the initial topic, but this claims to solve #70336 and my argument is about that part. If more appropriate, it could be discussed elsewhere.

Grisu, like Dragonbox, aim to convert a binary-coded floating points to a decimal representation which is correct (in the sense mentioned earlier, the parsed decimal representation yields the same binary coding). Here the coding is typically IEEE-754, with a current focus on double precision.

The above conclusion "thus should be rounded down when rounding to 1 decimal place" seems strange to me, because there is no way to assert that.

For example, 0.35 is coded like this:

6305039478318694*2^-54 or 0x16666666666666*2^0xffca => 0.34999999999999997779553950749686919152736663818359375

The value has been rounded to the 54-bit normalized mantissa using the round-even method, and indeed it's not a tie anymore (in decimal representation). If I increase the mantissa, I get this value:

6305039478318695*2^-54 or 0x16666666666667*2^0xffca => 0.350000000000000033306690738754696212708950042724609375

which is on the other side of the tie. So we see here that the error introduced when parsing "0.35" and initially rounding the value to code it as an f64 doesn't allow us to deduce on which side of the tie the initial value was.

Thus the operation that Dragonbox (or Grisu, Ryu, etc) does to minimize the output length while preserving the information and keeping a "correct" output, is not wrong. And it looks like it's more likely to be what the initial value was when it matters:

• if the initial value was 0.35, that is what we get
• if the initial value was 0.349999999999999978, likely as a result of a calculation, does it matter if the displayed value is "0.35"? No, because the error margin is the same. Likewise for rounding as "0.4" with {:.1} by assuming it was a tie.

At least that's how I understand the asymmetric problem of coding and displaying floating-point values respectively in base 2 and 10.

If we come back to the initial problem to solve, the current solution seems to be wrong in about 50% of the cases, which is a lot. So I'm only wondering whether there could be another problem than just the case of 0.5, but maybe in the scope of another code change.

I'll stop here, however, and sorry again if that was not the place.

I'm not sure what you mean by decimal representation here.

Do you agree that this is correct?

format!("{:.1}", 3152519739159347.0 / 9007199254740992.0) => "0.3"

I mean by that the decimal interpretation of the value which is coded in binary (IEEE-754). How the value is printed, if you will. Hopefully the first part of my post already clarified that. :)

I don't see the point of the last question.