Issue 22444: Floor divide should return int (original) (raw)

Issue22444

Created on 2014-09-19 16:56 by belopolsky, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (24)
msg227107 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-19 16:56
PEP 3141 defines floor division as floor(x/y) and specifies that floor() should return int type. Builtin float type has been made part of the PEP 3141 numerical tower, but floor division of two floats still results in a float. See also: * #1656 - Make math.{floor,ceil}(float) return ints per PEP 3141 * #1623 - Implement PEP-3141 for Decimal * https://mail.python.org/pipermail/python-ideas/2014-September/029392.html
msg227149 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2014-09-20 09:30
Is this change compelling enough to break compatibility, or is it just a matter of purity?
msg227151 - (view)	Author: Stefan Krah (skrah) *	Date: 2014-09-20 10:49
Perhaps it's worth mentioning that several people on Python-ideas took the opposite view: math.floor() should return a float. PEP-3141 does not mention Infinities and NaNs: "The Real ABC indicates that the value is on the real line, and supports the operations of the float builtin. Real numbers are totally ordered except for NaNs (which this PEP basically ignores)." Floats, however, are on the extended real number line, so we have a problem. :) Other languages =============== The PEP says that inspiration came from Scheme and Haskell. However, Scheme returns floats: ------------------------------- https://mail.python.org/pipermail/python-ideas/2014-September/029432.html Haskell seems to return the highest representable integer: ---------------------------------------------------------- Prelude> floor (1/0) 179769313486231590772930519078902473361797697894230657273430081157732675805500963132708477322407536021120113879871393357658789768814416622492847430639474124377767893424865485276302219601246094119453082952085005768838150682342462881473913110540827237163350510684586298239947245938479716304835356329624224137216 However, Haskell float support looks sketchy: --------------------------------------------- Prelude> floor (0/0) -269653970229347386159395778618353710042696546841345985910145121736599013708251444699062715983611304031680170819807090036488184653221624933739271145959211186566651840137298227914453329401869141179179624428127508653257226023513694322210869665811240855745025766026879447359920868907719574457253034494436336205824 Prelude> let x = 1 / 0 Prelude> x Infinity Prelude> x / 0 Infinity Considering the last two examples, I think Haskell should not provide any guidance here. ;)
msg227152 - (view)	Author: Stefan Krah (skrah) *	Date: 2014-09-20 10:54
Argh, forget the second Haskell example: inf / 0 is fine.
msg227156 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-20 16:01
> Is this change compelling enough to break compatibility, > or is it just a matter of purity? According to the PEP 3141, Integer is a subtype of Real, so one should be able to substitute an Integer whenever Real is expected. The reverse is not true, for example >>> [1,2][1.0] Traceback (most recent call last): File "", line 1, in TypeError: list indices must be integers, not float This is one of the common uses of floor division - to find an index of a cell in a regular grid: (x - start) // step. In this situation, it is convenient to have the result ready to be used as an index without a cast. The substitution principle also suggests that compatibility issues are likely to be small: in most contexts integers behave the same as floats or "better". Here is an example of a "better" behavior: >>> x = 1 + 10**50 >>> x * 1 == x True >>> x * 1.0 == x False The only case I can think of where float result may be preferable is inf // 1 because integers cannot represent infinity. However, this case is arguably already broken. What are the use-cases for float // float where integer result is not acceptable?
msg227158 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-20 16:06
> However, Scheme returns floats Does Scheme's default integer type support arbitrarily large values?
msg227159 - (view)	Author: Case Van Horsen (casevh)	Date: 2014-09-20 16:23
> > Does Scheme's default integer type support arbitrarily large values? > Yes, at least is does on the version I tested.
msg227161 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-20 16:38
> Perhaps it's worth mentioning that several people on Python-ideas > took the opposite view: math.floor() should return a float. I sympathize with the idea that math module functions should return floats. I find it unfortunate that math.floor delegates to the __floor__ dunder on non-floats instead of doing math.floor(x.__float__()). It would be more natural to have a floor builtin that *always* delegates to __floor__ and keep math a pure float library. Note that math module provides the means to compute C-style floor: >>> x = float('inf') >>> math.modf(x)[1] inf >>> x = -3.4 >>> math.modf(x)[1] -3.0 Maybe we should add floorf, ceilf, etc. as well. This, however, is a different issue from the one at hand here.
msg227162 - (view)	Author: Case Van Horsen (casevh)	Date: 2014-09-20 16:39
> What are the use-cases for float // float where integer result is not acceptable? It can lead to unexpected memory consumption when dealing with arbitrary precision values. What should Decimal('1e123456')//1 return? The result is exactly equal to Decimal('1e123456') but the corresponding Python integer will consume ~55KB of RAM. I'm also concerned that returning a very large integer will lead users to assume that the result is more precise than it really is. Assuming standard 64-bit double format, only the first 53 bits are significant. All the remaining bits are 0. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue22444> > _______________________________________
msg227163 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-20 16:44
> What should Decimal('1e123456')//1 return? I think Decimal case should be considered separately. Note that unlike float, they are not part of the numerical tower, so PEP 3141 arguments don't apply: >>> isinstance(1.0, numbers.Real) True >>> isinstance(decimal.Decimal(1), numbers.Real) False
msg227164 - (view)	Author: Case Van Horsen (casevh)	Date: 2014-09-20 17:05
On Sat, Sep 20, 2014 at 9:38 AM, Alexander Belopolsky <report@bugs.python.org> wrote: > > Alexander Belopolsky added the comment: > >> Perhaps it's worth mentioning that several people on Python-ideas >> took the opposite view: math.floor() should return a float. > > I sympathize with the idea that math module functions should return floats. I find it unfortunate that math.floor delegates to the __floor__ dunder on non-floats instead of doing math.floor(x.__float__()). It would be more natural to have a floor builtin that *always* delegates to __floor__ and keep math a pure float library. +1 > > Note that math module provides the means to compute C-style floor: > >>>> x = float('inf') >>>> math.modf(x)[1] > inf >>>> x = -3.4 >>>> math.modf(x)[1] > -3.0 That's not immediately obvious... > > Maybe we should add floorf, ceilf, etc. as well. This, however, is a different issue from the one at hand here. > i think the issues are related. PEP-3141 defines x//y as int(floor(x/y)). It also defines divmod(x, y) as (x//y, x % y). These definitions cannot all be satisfied at the same Python's divmod function takes extra effort to calculate x//y precisely. Those corrections are not possible via floor(). I maintain gmpy2 which wraps the GMP, MPFR, and MPC arbitrary precision libraries. I originally implemented x//y as floor(x/y). That choice lead to errors in divmod() that I've fixed in the development version. I still need to fix floor division: do I make it compatible with divmod() or floor()? My preference would be to define floor division and divmod in terms of each other and allow math.ceil()/floor()/trunc() to return floating point values. The current definitions are impossible to satisfy. I mentioned my concerns about memory growth in another comment. I'm not as concerned about the unexpected memory growth in floor division as I am in floor() etc.
msg227165 - (view)	Author: Case Van Horsen (casevh)	Date: 2014-09-20 17:13
>> What should Decimal('1e123456')//1 return? > > I think Decimal case should be considered separately. Note that unlike float, they are not part of the numerical tower, so PEP 3141 arguments don't apply: > >>>> isinstance(1.0, numbers.Real) > True >>>> isinstance(decimal.Decimal(1), numbers.Real) > False > I maintain gmpy2 and I've had requests to support the numeric tower. gmpy2 has integral, rational, real, and complex types so I should be able to.
msg227166 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2014-09-20 17:49
> What are the use-cases for float // float where integer result is not acceptable? I can't think of any. I was mostly making the case for conservatism here. The indexing use case is interesting, although I suppose enumerate() should eliminate most instances of it.
msg227172 - (view)	Author: Raymond Hettinger (rhettinger) *	Date: 2014-09-20 20:14
> Is this change compelling enough to break compatibility, > or is it just a matter of purity? I agree with Antoine that making this change is a really bad idea. 1) The current behavior has been around for a long time and is implemented in several modules including decimal and fractions. As core devs, we need to keep focused on a priority of making the language stable (not making changes that truly necessary and invalidating all previously published material) and more importantly not adding yet more obstacles to converting from Python 2 to Python 3 (which Guido has called "death by a thousand cuts"). 2) The current behavior can be useful it that it allows floor division operations without unexpected type conversions occurring in the middle of an expression. We really don't want to break those use cases. # Here is a simple example of a chain of calculations # where preserving the type matters from __future__ import print_function from fractions import Fraction from decimal import Decimal def f(x, y): return x // 3 * 5 / 7 + y def g(x, y): return int(x // 3) * 5 / 7 + y for x, y in [ (Fraction(85, 7), Fraction(2, 3)), (Decimal('12.143'), Decimal('0.667')), (12.143, 0.667), ]: print(f(x, y), g(x, y)) In Python 2: ------------ 8/3 8/3 3.524142857142857142857142857 2.667 3.52414285714 2.667 In Python 3: ------------ 3.5238095238095237 3.5238095238095237 Traceback (most recent call last): ... return int(x // 3) * 5 / 7 + y TypeError: unsupported operand type(s) for +: 'float' and 'decimal.Decimal' I am a strong -1 against breaking code that relies on the floor division being type preserving. The PEP should be revised to say that floor division is defined to return a value that is *equal* to an Integral but not place any restriction on the return type.
msg227180 - (view)	Author: Tim Peters (tim.peters) *	Date: 2014-09-20 21:48
Floor division on floats is an unattractive nuisance and should be removed, period - so there ;-) But short of that, I favor leaving it alone. Whose life would be improved by changing it to return an int? Not mine - and doing so anyway is bound to break existing code. +1 on changing the PEP instead (as Raymond suggested).
msg227183 - (view)	Author: Alex Gaynor (alex) *	Date: 2014-09-20 22:16
I can't say that I've ever used // on floats, but it seems to me anyone doing so (as opposed to normal division + explicit rounding) *intentionally* might be broken by this change, but anyone doing this incidentally is not really in a "gotcha" situation. Since this is a type-specific behavior, and not a value-specific one, I don't really think there's a win in changing the behavior, and staying backwards compatible is much better.
msg227277 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-22 15:39
[Raymond] > The current behavior has been around for a long time and is implemented in several modules including decimal and fractions. No, in the fractions module floor division returns an int: >>> type(Fraction(2) // Fraction(1)) <class 'int'> It is also implemented in the datetime module where >>> type(timedelta(2) // timedelta(1)) <class 'int'> [Raymond] # Here is a simple example of a chain of calculations # where preserving the type matters .. def f(x, y): return x // 3 * 5 / 7 + y def g(x, y): return int(x // 3) * 5 / 7 + y [/Raymond] I am not sure what is the problem here. In Python 3: >>> f(12.143, 0.667) 3.5241428571428575 >>> g(12.143, 0.667) 3.5241428571428575
msg227279 - (view)	Author: Mark Dickinson (mark.dickinson) *	Date: 2014-09-22 17:00
-1 from me, too. It's an unnecessary change, and the conversion from float to integer potentially expensive compared to the computation of the floating-point result (especially in extended floating-point implementations that allow a wider exponent range). If this is about consistency between `//` and `math.floor`, I'd much rather see `math.floor` go back to returning a float instead of an int.
msg227281 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-22 17:23
[Raymond] > The PEP should be revised to say that floor division is defined to > return a value that is *equal* to an Integral but not place any > restriction on the return type. If we take this route, what float('inf') // 1 and float('nan') // 1 should return?
msg227282 - (view)	Author: Mark Dickinson (mark.dickinson) *	Date: 2014-09-22 17:29
> If we take this route, what float('inf') // 1 and float('nan') // 1 should return? Probably exactly the same as they do right now. I think there's an argument that `float('inf') // 1` "should have been" `float('inf')`. But I'm not sure there's much of a case for *changing* the current return value.
msg227301 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-22 19:40
Mark, Raymond suggested that "The PEP 3141 should be revised to say that floor division is defined to return a value that is *equal* to an Integral". Since nan or inf are not *equal* to any Integral, the current implementation does not comply. In the absence of a recommendation in the PEP, implementers of new numeric types are left with little guidance because existing types are inconsistent: >>> Decimal('inf') // 1 Decimal('Infinity') >>> float('inf') // 1 nan
msg227303 - (view)	Author: Stefan Krah (skrah) *	Date: 2014-09-22 19:48
Alexander Belopolsky <report@bugs.python.org> wrote: > Raymond suggested that "The PEP 3141 should be revised to say that floor division is defined to return a value that is *equal* to an Integral". I guess it should say "equal to an Integral or a special value". > Since nan or inf are not *equal* to any Integral, the current implementation does not comply. In the absence of a recommendation in the PEP, implementers of new numeric types are left with little guidance because existing types are inconsistent: > > >>> Decimal('inf') // 1 > Decimal('Infinity') > >>> float('inf') // 1 > nan I think both should return inf.
msg227305 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2014-09-22 19:55
skrah> I think both should return inf. What about this case: >>> Decimal('1') // Decimal('-inf') Decimal('-0') >>> 1. // float('-inf') -1.0
msg227306 - (view)	Author: Mark Dickinson (mark.dickinson) *	Date: 2014-09-22 20:02
I think that one's covered by #22198.