: [Python-Dev] RE: Painful death in debug build (original) (raw)

Tim Peters tim.one@home.com
Sat, 6 Oct 2001 04:35:01 -0400

[Tim]

This is enough to reproduce the problem under a debug Windows build:

--------------------------------------------------------------- class X(long): pass x = X(0xffffL) print "before del" del x print "after del" --------------------------------------------------------------- It does not fail if the instance is created via x = X(0x7fffL) instead. It does fail on x = X(0x8000L) The relevant difference is that Python uses 15-bit "digits" internally for longs, so 0x8000 may be the smallest value that will cause it to allocate more memory than already comes for free with the longobject header

Now I'm in:

PyObject * PyType_GenericAlloc(PyTypeObject *type, int nitems) { #define PTRSIZE (sizeof(PyObject *))

int size;
PyObject *obj;

/* Inline PyObject_New() so we can zero the memory */

That comment doesn't seem to make much sense; the code following does a whole bunch that PyObject_New() doesn't do.

size = _PyObject_VAR_SIZE(type, nitems);

Then size = basicsize + 2 * itemsize = 26 + 2*2 = 30.

/* Round up size, if necessary, so we fully zero out __dict__ */
if (type->tp_itemsize % PTRSIZE != 0) {
    size += PTRSIZE - 1;
    size /= PTRSIZE;
    size *= PTRSIZE;
}

I don't understand what that comment is trying to say; the code following it bumps size up to 32, and that turns out to be "the problem":

if (PyType_IS_GC(type)) {
    obj = _PyObject_GC_Malloc(type, nitems);
}
else {
    obj = PyObject_MALLOC(size);
}

The subtype is a GC type, so _PyObject_GC_Malloc allocates

sizeof(PyGC_Head) + (size_t)_PyObject_VAR_SIZE(tp, size) = 12 + 30 =

42 bytes. Note that the second term is the 30 originally computed for size, not the 32 that size was later bumped up to.

if (obj == NULL)
    return PyErr_NoMemory();

Nope.

memset(obj, '\0', size);

Bingo! obj is at offset 12 from the allocated block, and size is 32, so we zero out the last 32 of the trailing 30 (=42-12) bytes allocated; i.e., we zero out two bytes beyond the end of the allocated block. Everything else follows from that. Seems pretty clear that this doesn't have anything specific to do with longs. Yup:

class X(str): ... pass ... [6960 refs] x = X("ab") [6969 refs] x = 1

That also blows up.

Since I'm not sure what the roundup of size is trying to accomplish, I'm not sure what to do about it. If I ignore the comment, I imagine we want subtype pointer fields (like tp_dict) following the base-type prefix to be naturally aligned for the architecture, and then padding makes sense to me.