[Python-Dev] Suggested memory API rules for 2.3 (original) (raw)

Guido van Rossum guido@python.org
Wed, 03 Apr 2002 11:56:56 -0500

"Programs" does not include the Python core. We can do anything we need to do in the core, based on our complete knowledge of each release's internal implementation details.

However, since many people use the core as an example, we may have to be careful here. Certainly xxmodule.c, xxobject.c and xxsubtype.c should do the recommended thing; but if too many of the built-in objects use the macros despite those being in danger of deprecation, that might still perpetuate use of those macros.

Mixing and Matching

After memory has been obtained via one of these functions, it should be resized and freed only by a function from the same line, except that PyMemFree may be used freely in place of PyMemDel, and PyObjectFree in place of PyObjectDel.

I actually had a slightly different rule in mind: mixing and matching between two adjacent lines that only differ by case is also permitted. Except of course that the macro lines will be deprecated, so maybe it doesn't matter to spell this out.

Maintaining the Free/Del pseudo-distinction is pointless.

For backward compatibility, memory obtained via the object memory family can be freed by any of Py{Mem, Object}{Free, FREE, Del, DEL}. Mixing functions from the object family with the raw memory family is likely to become deprecated, Memory obtained by PyMem{Malloc, MALLOC} shall be resized only by PyMem{Realloc, REALLOC}. Memory obtained by PyMem{New, NEW} shall be resized only by PyMem{Resize, RESIZE}. Memory obtained by PyObject{Malloc, MALLOC} shall be resized only by PyObject{Realloc, REALLOC}. Note that the eight ways to spell "free" all have to map to the pymalloc free when pymalloc is enabled in 2.3. There is no way to spell "give me the raw platform free(), damn it", except for "free". If we think it's important to have a such a way in the core, it should be added to the private API.

IMO, PyMem_Malloc() and PyMem_Free() should be good enough.

Relationship to Platform Allocator

All names in lines A, B, C and D ultimately invoke the platform C malloc, realloc, or free. However, programs shall not mix any of these names with direct calls to the platform malloc, calloc, realloc, or free referencing the same base memory addresses, as Python may need to perform bookkeeping of its own around its calls to the platform allocator.

Really? Why not just say these are wrappers around malloc and free? (On platforms where it matters, they will be guaranteed to use the same heap as the core uses -- this is apparently an issue with Windows DLLs.)

--Guido van Rossum (home page: http://www.python.org/~guido/)