Issue 36655: Division Precision Problem (original) (raw)
Issue36655
Created on 2019-04-18 08:25 by kulopo, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Messages (3) | ||
|---|---|---|
| msg340471 - (view) | Author: kulopo (kulopo) | Date: 2019-04-18 08:25 |
| >>> a=224847175712806907706081280 >>> b=4294967296 >>> assert int(a*b/b)==int(a) Traceback (most recent call last): File "", line 1, in AssertionError (a can be exact divided by b) | ||
| msg340472 - (view) | Author: Christian Heimes (christian.heimes) *  |
Date: 2019-04-18 08:30 |
| This is the expected and correct behavior. Python's float are IEEE 754 floats, https://en.wikipedia.org/wiki/IEEE_754. IEE 754 have a limited precision. 224847175712806907706081280 / 4294967296 is not exactly dividable under IEEE 754 semantics. >>> a=224847175712806907706081280 >>> b=4294967296 >>> a/b 5.235131264496755e+16 | ||
| msg340473 - (view) | Author: Eric V. Smith (eric.smith) * |
Date: 2019-04-18 08:32 |
| Also see https://docs.python.org/3/tutorial/floatingpoint.html for some Python-specific details. |
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:14 | admin | set | github: 80836 |
| 2019-04-18 08:32:19 | eric.smith | set | nosy: + eric.smithmessages: + |
| 2019-04-18 08:30:33 | christian.heimes | set | status: open -> closednosy: + christian.heimesmessages: + resolution: not a bugstage: resolved |
| 2019-04-18 08:25:15 | kulopo | create |