| msg101887 - (view) |
Author: (owirj) |
Date: 2010-03-29 12:29 |
| python -V: Python 2.6.5 Executing on 32-bit machine. From documentation ( http://docs.python.org/reference/expressions.html ), "5.7. Shifting operations": "These operators accept plain or long integers as arguments. The arguments are converted to a common type. They shift the first argument to the left or right by the number of bits given by the second argument." In interpreter: >>> x = 2677691728509L << 2147483648L Traceback (most recent call last): File "", line 1, in OverflowError: long int too large to convert to int hint: 2147483648 = (1 << 31) First argument is long, second is long too, so expected behavior was normal execution. But as seen from the code above interpreter gives overflow error. Looks like he tries to convert second argument to int type, because this code works gracefully: >>> x = 2677691728509L << 2147483647L |
|
|
| msg101889 - (view) |
Author: R. David Murray (r.david.murray) *  |
Date: 2010-03-29 13:13 |
| This appears to be working as designed, and so is probably a doc issue. Python 3, which only has one integer type, has the same behavior: >>> x = 2677691728509 << 2147483648 Traceback (most recent call last): File "", line 1, in OverflowError: Python int too large to convert to C long Adding Mark Dickinson as nosy to get his opinion on whether this is a doc bug or a feature request :) |
|
|
| msg101891 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2010-03-29 14:55 |
| I get another error on a 64-bit build: >>> x = 2677691728509L << 2147483648L Traceback (most recent call last): File "", line 1, in ValueError: outrageous left shift count (yes, Python is outraged by such a large amount of left shifting) The reason you get an OverflowError in 32-bit mode is that 2**31 is too large to be represented as a (signed) C long, rather than unsigned. |
|
|
| msg101893 - (view) |
Author: (owirj) |
Date: 2010-03-29 15:26 |
| Antoine Pitrou: The reason you get an OverflowError in 32-bit mode is that 2**31 is too large to be represented as a (signed) C long, rather than unsigned. I understand that, but after reading documentation, was expecting it to convert second argument to long type, which is not concerned with architecture dependant implementation of int. |
|
|
| msg102159 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2010-04-02 11:29 |
| The original error, for a 32-bit machine, looks like expected behaviour to me; at worst, it's a doc bug. If the right-hand argument doesn't fit into a Py_ssize_t, then I think it's reasonable to refuse to do the shift. But Antoine's 'outrageous left shift count' comes from the following code, in long_lshift in Objects/longobject.c. if ((long)(int)shiftby != shiftby) { PyErr_SetString(PyExc_ValueError, "outrageous left shift count"); goto lshift_error; } I think this code dates from the days when the number of 'digits' of a PyLong was held in an int rather than a Py_ssize_t, and I think it's a (minor) bug. A 64-bit machine with sufficient RAM shouldn't mind shifting something by >= 2**31 places. |
|
|
| msg102473 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2010-04-06 16:26 |
| Patch that unoutrages Python attached: it allows shift counts of up to sys.maxsize in both left shift and right shift. I don't have a test for this, since the only tests I can think of (e.g. actually doing 1 << (2**31)) require > 270Mb of RAM, and that may be too much for some of the buildbots. (We could add a 'largemem' special resource to regrtest, I suppose, but it doesn't seem worth the effort just for this.) This patch doesn't solve the OPs problem, though. The PyLong code *could* be modified to accept larger shift counts (e.g., up to 8*PY_SSIZE_T_MAX; beyond this you'll get a MemoryError anyway), but this seems like a significant effort for little gain. It might make more sense for right shifting, though. Anyway, I'd definitely call that a feature request. |
|
|
| msg102475 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2010-04-06 16:53 |
| Patch applied in r79843 (and added forgotten Misc/NEWS entry in r79844); merged to py3k in r79845. |
|
|
| msg102480 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2010-04-06 18:24 |
| By the way, this has nothing to do with int versus long (unless I'm misunderstanding the original issue); it's simply that large shift counts cause an OverflowError. I've added a note about this to the docs in r79852. |
|
|