[Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Jun 20 12:16:56 CEST 2013

* Add new GIL-free (no need to hold the GIL) memory allocator functions:

- void* PyMemRawMalloc(sizet size) - void* PyMemRawRealloc(void *ptr, sizet newsize) - void PyMemRawFree(void *ptr) - the behaviour of requesting zero bytes is not defined: return NULL or a distinct non-NULL pointer depending on the platform. (...) * Add new functions to get and set internal functions of PyMemMalloc(), PyMemRealloc() and PyMemFree(): - void PyMemGetAllocator(PyMemBlockAllocator *allocator) - void PyMemSetAllocator(PyMemBlockAllocator *allocator) - malloc(ctx, 0) and realloc(ctx, ptr, 0) must not return NULL: it would be treated as an error. - default allocator: malloc(), realloc(), free(); PyMemMalloc(0) calls malloc(1) and PyMemRealloc(NULL, 0) calls realloc(NULL, 1)

Oh, one more question: PyMem_RawMalloc(0) has an undefined behaviour, whereas PyMem_Malloc(0) has a well defined behaviour (don't return NULL). Adding "if (size == 1) size = 0;" in the default implementation of PyMem_RawMalloc(0) should not have a visible overhead, but it gives the same guarantee than PyMem_Malloc(0) (don't return NULL). Do you agree to add the test?

I chose to implement "if (size > (size_t)PY_SSIZE_T_MAX) return NULL;" in Py*_Malloc(), whereas "if (size == 0) size =1;" is implemented in the inner function (_PyMem_Malloc). An application may use an allocator which has already a well defined behaviour (a "malloc(0)" that don't return NULL) and I expect malloc(1) to allocate "more" memory than malloc(0) (malloc(0) may create a singleton) :-)

Victor

More information about the Python-Dev mailing list