[Python-Dev] Debug entry points for PyMalloc (original) (raw)

Tim Peters tim.one@comcast.net
Thu, 21 Mar 2002 01🔞24 -0500

The thing I've dreaded most about switching to pymalloc is losing the invaluable memory-corruption clues supplied by the Microsoft debug-build malloc. On more than one occasion, they've found wild stores, out-of-bounds reads, reads of uninitialized memory, and reads of free()ed memory in Python. It does this by spraying special bytes all over malloc'ed memory at various times, then checking the bytes for sanity at free() and realloc() times.

This kind of stuff is going to be pure hell under pymalloc, because there's no padding at all between chunks pymalloc passes out, and pymalloc stores valid addresses at the start of free()ed memory. So a wild store probably can't be detected as any sort of memory corruption, it will simply overwrite part of some other end-user object -- or corrupt pymalloc's internal pointers linking free()ed memory (and pymalloc simply goes nuts then). Several months ago Martin and I took turns thinking about a memory overwrite problem in the Unicode stuff that showed up under pymalloc, and it was indeed pure hell to track it down.

Following is a sketch for teaching pymalloc how to do something similar to the MS scheme. A twist over the MS scheme is adding a "serial number" to the pad bytes, incremented by one for each malloc/realloc. At a crude level, this gives a sense of age to the eyeball; for reproducible memory nightmares, it gives an exact way to set a data, or counting, breakpoint (on the next run) to capture the instant at which a doomed-to-go-bad memory block first gets passed out. I hope that addresses the worst problem the MS scheme still leaves untouched: you can catch memory corruption pretty will with it, but all you know then is that "the byte at this address is bad", and you have no idea what the memory's original purpose in life was.

Sketch of Debug Mode for PyMalloc

DL_IMPORT(void *) _PyMalloc_DebugMalloc(size_t nbytes); DL_IMPORT(void *) _PyMalloc_DebugRealloc(void *p, size_t nbytes); DL_IMPORT(void) _PyMalloc_DebugFree(void *p);

#ifdef WITH_PYMALLOC DL_IMPORT(void *) _PyMalloc_Malloc(size_t nbytes); DL_IMPORT(void *) _PyMalloc_Realloc(void *p, size_t nbytes); DL_IMPORT(void) _PyMalloc_Free(void *p);

DL_IMPORT(void *) _PyMalloc_DebugMalloc(size_t nbytes); DL_IMPORT(void *) _PyMalloc_DebugRealloc(void *p, size_t nbytes); DL_IMPORT(void) _PyMalloc_DebugFree(void *p);

#ifdef PYMALLOC_DEBUG #define _PyMalloc_MALLOC _PyMalloc_DebugMalloc #define _PyMalloc_REALLOC _PyMalloc_DebugRealloc #define _PyMalloc_FREE _PyMalloc_DebugFree

#else /* WITH_PYMALLOC && !PYMALLOC_DEBUG */ #define _PyMalloc_MALLOC _PyMalloc_Malloc #define _PyMalloc_REALLOC _PyMalloc_Realloc #define _PyMalloc_FREE _PyMalloc_Free

#endif /* PYMALLOC_DEBUG */

#else /* !WITH_PYMALLOC / #define _PyMalloc_MALLOC PyMem_MALLOC #define _PyMalloc_REALLOC PyMem_REALLOC #define _PyMalloc_FREE PyMem_FREE #endif / WITH_PYMALLOC */

#define PYMALLOC_CLEANBYTE 0xCB /* uninitialized memory / #define PYMALLOC_DEADBYTE 0xDB / free()ed memory / #define PYMALLOC_FORBIDDENBYTE 0xFB / unusable memory */

The debug malloc/free/realloc use these as follows. Note that this stuff is done regardless of whether PyMalloc handles the request directly or passes it on to the platform malloc (in fact, the debug entry points won't know and won't care).