[Python-Dev] Bad interaction of index and sequence repeat (original) (raw)

Guido van Rossum guido at python.org
Fri Jul 28 18:05:57 CEST 2006

Argh.

I also find it a bug.

I also feel responsible because I reviewed the patch. :-(

In my recollection I tried to avoid this exact behavior. I wanted index() to just return the unclipped int or long value, but have a C API that clipped it for use in slice operations. It looks like I failed (the patch went through so many revisions that at some point I must've stopped caring).

--Guido

On 7/28/06, Nick Coghlan <ncoghlan at gmail.com> wrote:

David Hopwood wrote: > Armin Rigo wrote: >> Hi, >> >> There is an oversight in the design of index() that only just >> surfaced :-( It is responsible for the following behavior, on a 32-bit >> machine with >= 2GB of RAM: >> >> >>> s = 'x' * (2**100) # works! >> >>> len(s) >> 2147483647 >> >> This is because PySequenceRepeat(v, w) works by applying w.index in >> order to call v->sqrepeat. However, index is defined to clip the >> result to fit in a Pyssizet. > > Clipping the result sounds like it would never be a good idea. What was > the rationale for that? It should throw an exception.

A simple demonstration of the clipping behaviour that works on machines with limited memory: >>> (2**100).index() 2147483647 >>> (-2**100).index() -2147483648 PEP 357 doesn't even mention the issue, and the comment on longindex in the code doesn't give a rationale - it just notes that the function clips the result. Neither the PyNumberAsIndex nor the index documentation mention anything about the possibility of clipping, and there's no test case to verify this behaviour. I'm inclined to call it a bug, too, but I've cc'ed Travis to see if he can shed some light on the question - the implementation of longindex explicitly suppresses the overflow error generated by longasssizet, so the current behaviour appears to be deliberate. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list