[Python-Dev] Moving forward on the object memory API (original) (raw)

Guido van Rossum guido@python.org
Sun, 31 Mar 2002 16:07:59 -0500

It looks like Tim managed to work some magic and make pymalloc's free handle memory allocated by either the system allocator or pymalloc itself. That means we can make PyMemDEL use the pymalloc's 'free' and make PyObjectNEW use pymalloc's 'malloc'. Here's my plan for making this happen:

#define PyMemMALLOC(n) malloc(n) #define PyMemREALLOC(p, n) realloc((void *)(p), (n)) #define PyMemFREE(p) free((void *)(p))

+1

I think making PyMemFREE call free() is safe. I haven't seen any extension modules allocate with PyObjectNEW and free with PyMemFREE. We deprecate the PyMem* functions. There's no need for them, IMHO:

#define PyMemMalloc PyMemMALLOC #define PyMemNew PyMemNEW #define PyMemResize PyMemRESIZE #define PyMemFree PyMemDEL #define PyMemDel PyMemDEL /* See comment near MALLOCZERORETURNSNULL in pyport.h. */ _#define PyMemRealloc(p, n) _ _do { _ _sizet n = n; _ _PyMemREALLOC(p, n ? n : 1); _ } while (0)

+1

Next, we base PyObject{MALLOC,REALLOC,FREE} on pymalloc. Basically:

#ifdef WITHPYMALLOC #define PyObjectMALLOC(n) PyMallocMalloc(n) #define PyObjectREALLOC(op, n) PyMallocRealloc((void *)(op), (n)) #define PyObjectFREE(op) PyMallocFree((void *)(op)) #else #define PyObjectMALLOC(n) PyMemMALLOC(n) #define PyObjectREALLOC(op, n) PyMemREALLOC((void *)(op), (n)) #define PyObjectFREE(op) PyMemFREE((void *)(op)) #endif

Couldn't these always use the first set of definitions? I suppose that when configured --without-pymalloc, _PyMalloc_Malloc and friends are aliases for malloc?

Also, do we need these at all? (Hm, I must be missing something, but I can't figure what.)

PyMemDEL and PyMemFree need to call pymalloc:

#define PyMemDEL(op) PyObjectFREE(op)

Why not _PyMalloc_Malloc?

We go through the source and replace all the PyMem* function calls with the equivalent PyMem* macro calls and replace PyMemDEL with PyMemFREE.

Or PyObject_Del (depending on the situation)?

The recommended API for extension modules writers would be PyMem{MALLOC,REALLOC,NEW,RESIZE,FREE}, PyObject{New,NewVar,Del}, and PyObjectGC{New,NewVar,Del}.

Is there a difference between PyMem_MALLOC and PyMem_NEW? Between PyMem_REALLOC and PyMem_RESIZE? If there isn't, shouldn't we recommend one set and deprecate the other? I think I like the NEW family best (it's got seniority).

I suppose the patches that changed PyObject_New into PyMalloc_New can be reverted now?

Finally, there still seem to be too many macros and functions. But it's better than before!

--Guido van Rossum (home page: http://www.python.org/~guido/)