[Numpy-discussion] integer power in scalarmath: how to handle overflow? (original) (raw)
David Douard david.douard at logilab.fr
Tue Jun 6 04:44:20 EDT 2006
- Previous message (by thread): [Numpy-discussion] integer power in scalarmath: how to handle overflow?
- Next message (by thread): [Numpy-discussion] integer power in scalarmath: how to handle overflow?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On Mon, Jun 05, 2006 at 05:10:23PM -0400, David M. Cooke wrote:
I just ran into the fact that the power function for integer types isn't handled in scalarmath yet. I'm going to add it, but I'm wondering what people want when power overflows the integer type?
Taking the concrete example of a = uint8(3), b = uint8(10), then should a ** b return 1) the maximum integer for the type (255 here) 2) 0 3) upcast to the largest type that will hold it (but what if it's larger than our largest type? Return a Python long?) 4) do the power using "long" like Python does, then downcast it to the type (that would return 169 for the above example) 5) something else? I'm leaning towards #3; if you do a ** 10, you get the right answer (59049 as an int64scalar), because 'a' is upcasted to int64scalar, since '10', a Python int, is converted to that type. Otherwise, I would choose #1.
I agree, #1 seems the better solution for me.
BTW, I'm quite new on this list, and I don't know is this has already been discussed (I guess I has): why does uint_n arithmetics are done in the Z/(2**n)Z field (not sure about the maths correctness here)? I mean:
a = numpy.uint8(10) a*a 100 aaa # I'd like to have 255 here 232 1000%256 232
It would be really a nice feature to be able (by the mean of a numpy flag or so) to have bound limited uint operations (especially when doing image processing...).
David
-- David Douard LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian : http://www.logilab.fr/formations Développement logiciel sur mesure : http://www.logilab.fr/services Informatique scientifique : http://www.logilab.fr/science -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20060606/9718c8cc/attachment.sig>
- Previous message (by thread): [Numpy-discussion] integer power in scalarmath: how to handle overflow?
- Next message (by thread): [Numpy-discussion] integer power in scalarmath: how to handle overflow?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]