[Python-Dev] can this overflow (list insertion)? (original) (raw)

Tim Peters tim_one@email.msn.com
Sat, 12 Aug 2000 19:37:14 -0400

[Trent Mick]

from Objects/listobject.c:

static int ins1(PyListObject *self, int where, PyObject *v) { ... self->obsize++; <-------------- can this overflow? return 0; }

... Is it or was this relying on sizeof(obsize) == sizeof(void*), hence a list being able to hold as many items as there is addressable memory?

I think it's more relying on the product of two other assumptions: (a) sizeof(int) >= 4, and (b) nobody is going to make a list with 2 billion elements in Python. But you're right, sooner or later that's going to bite us.

proposed patch:

*** python/dist/src/Objects/listobject.c Fri Aug 11 16:25:08 2000 --- Python/dist/src/Objects/listobject.c Fri Aug 11 16:25:36 2000 *************** *** 149,155 **** PyINCREF(v); items[where] = v; self->obitem = items; ! self->obsize++; return 0; } --- 149,159 ---- PyINCREF(v); items[where] = v; self->obitem = items; ! if (self->obsize++ == INTMAX) { ! PyErrSetString(PyExcOverflowError, ! "cannot add more objects to list"); ! return -1; ! } return 0; }

+1 on catching it, -1 on this technique. You noted later that this will make trash out of ob_size if it triggers, but the list has already grown and been shifted by this time too, so it's left in an insane state (to the user, the last element will appear to vanish).

Suggest checking at the start of the routine instead:

   if (self->ob_size == INT_MAX) {
          PyErr_SetString(PyExc_OverflowError,
                  "cannot add more objects to list");
          return -1;
  }

Then the list isn't harmed.