[Python-Dev] Enhancement of Python memory allocators (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Jun 13 13:03:46 CEST 2013

2013/6/13 Nick Coghlan <ncoghlan at gmail.com>:

Yes, that sounds better. One of the biggest problems with the current startup sequence is the way it relies on environment variables for configuration, which makes life hard for other applications that want to embed the CPython runtime.

I wrote a new patch (attached to issue #3329) adding a new PyMem_SetupDebugHooks() function. So if an application replaces Python memory allocator functions, it can still can PyMem_SetupDebugHooks() to benefit of the Python debug hooks detecting memory bugs. The function does nothing if hooks are already installed or if Python is not compiled in debug mode. With this function, the API is now complete for all use cases.

The PEP 432 helps to configure embedded Python, but the new "Set" functions (ex: PyMem_SetAllocators) are still needed for my pytracemalloc module which installs hooks at runtime, when Python is fully initialized (the hooks can be installed anytime). pytracemalloc is just an example, you may use PyMem_SetAllocators for other debug or performance purpose.

With my patch, allocator functions like PyMem_Malloc() are no more macro, and are always the same function. This helps the stable ABI: C extension modules do not need to be recompiled to benefit of the debug hooks ;-)

Victor

More information about the Python-Dev mailing list