[Python-Dev] PyMem_Realloc corner case (original) (raw)

Tim Peters tim.one@comcast.net
Fri, 01 Mar 2002 02:27:25 -0500

[Jason Orendorff]

I just got bit by the following:

On Windows,

Which flavor of Windows? Which compiler? Which version of Python?

PyMemRealloc() doesn't seem to work as advertised in http://www.python.org/doc/current/api/memoryInterface.html

In particular, "if n is equal to zero, the memory block is resized but is not freed, and the returned pointer is non-NULL".

I expect Fred made that up . Whatever, it isn't true on all platforms.

In Objects/object.c, lines 1878-1896, PyMemMalloc() and PyMemRealloc() seem to try to guarantee this behavior. Somehow it isn't working for me.

What does "isn't working" mean? What were your inputs and what was your output? What did you expect instead?

Does this mean MALLOCZERORETURNSNULL should be defined in PC/pyconfig.h? Or do I have an off version of the CRT that causes problems nobody else has ?

Probably no and no: realloc'ing to 0 bytes is stupid, so I'd rather you stopped hitting yourself over the head and that Python stopped encouraging you to do so . You have control over one of those, anyway.

MS malloc(0) does not return NULL, which is why MALLOC_ZERO_RETURNS_NULL should not be #define'd. However, MS realloc(p, 0) returns NULL if and only if p is not NULL (well, MS realloc(NULL, 0) can return NULL if there's not enough heap space remaining to allocate a dummy object), and if p is not NULL does free the block. All of this is fine by the C standard.

If people think this needs to "be fixed", fine, but I woould strongly oppose #define'ing MALLOC_ZERO_RETURNS_NULL on Windows: adding the silly 1 to all malloc calls then can cause overallocation by 8 bytes, that's a horrible waste, and whoever assumed malloc(0) behavior was correlated with realloc(not-NULL, 0) behavior to begin with was wrong. Better to get rid of MALLOC_ZERO_RETURNS_NULL, and get rid of _PyMem_EXTRA, instead special-casing the snot out of a 0 argumentl e.g. instead of

void * PyMem_Realloc(void *p, size_t nbytes) { #if _PyMem_EXTRA > 0 if (nbytes == 0) nbytes = _PyMem_EXTRA; #endif return PyMem_REALLOC(p, nbytes); }

do

void * PyMem_Realloc(void *p, size_t nbytes) { return PyMem_REALLOC(p, nbytes ? nbytes : 1); }

instead.

Note: The pymalloc realloc works just like the MS realloc here, so it's not "just Windows"; from obmalloc.c's _THIS_REALLOC:

        /* Don't bother if a smaller size was requested
           except for realloc(p, 0) == free(p), ret NULL */
        if (nbytes == 0) {
            _THIS_FREE(p);
            bp = NULL;

In fact, I believe it's common for realloc(not-NULL, 0) to act like free(not-NULL), and that's allowed but not required by C. The overly gross MALLOC_ZERO_RETURNS_NULL is thus likely not defined on lots of platforms with this realloc behavior.

Can we call it a doc error?