[Python-Dev] Moving forward on the object memory API (original) (raw)

Neil Schemenauer nas@python.ca
Sun, 31 Mar 2002 11:08:13 -0800

It looks like Tim managed to work some magic and make pymalloc's free handle memory allocated by either the system allocator or pymalloc itself. That means we can make PyMem_DEL use the pymalloc's 'free' and make PyObject_NEW use pymalloc's 'malloc'. Here's my plan for making this happen:

#define PyMem_MALLOC(n)         malloc(n)
#define PyMem_REALLOC(p, n)     realloc((void *)(p), (n))
#define PyMem_FREE(p)           free((void *)(p))

I think making PyMem_FREE call free() is safe. I haven't seen any extension modules allocate with PyObject_NEW and free with PyMem_FREE. We deprecate the PyMem_* functions. There's no need for them, IMHO:

#define PyMem_Malloc PyMem_MALLOC
#define PyMem_New PyMem_NEW
#define PyMem_Resize PyMem_RESIZE
#define PyMem_Free PyMem_DEL
#define PyMem_Del PyMem_DEL
/* See comment near MALLOC_ZERO_RETURNS_NULL in pyport.h. */
#define PyMem_Realloc(p, n) \
        do { \
                size_t _n = n; \
                PyMem_REALLOC(p, _n ? _n : 1); \
        } while (0)

Next, we base PyObject_{MALLOC,REALLOC,FREE} on pymalloc. Basically:

#ifdef WITH_PYMALLOC
#define PyObject_MALLOC(n)      _PyMalloc_Malloc(n)
#define PyObject_REALLOC(op, n) _PyMalloc_Realloc((void *)(op), (n))
#define PyObject_FREE(op)       _PyMalloc_Free((void *)(op))
#else
#define PyObject_MALLOC(n)      PyMem_MALLOC(n)
#define PyObject_REALLOC(op, n) PyMem_REALLOC((void *)(op), (n))
#define PyObject_FREE(op)       PyMem_FREE((void *)(op))
#endif

PyMem_DEL and PyMem_Free need to call pymalloc:

#define PyMem_DEL(op) PyObject_FREE(op)

We go through the source and replace all the PyMem_* function calls with the equivalent PyMem_* macro calls and replace PyMem_DEL with PyMem_FREE.

The recommended API for extension modules writers would be PyMem_{MALLOC,REALLOC,NEW,RESIZE,FREE}, PyObject_{New,NewVar,Del}, and PyObject_GC_{New,NewVar,Del}.

Neil