[Python-Dev] Python initialization and embedded Python (original) (raw)

Victor Stinner victor.stinner at gmail.com
Mon Nov 20 17:03:50 EST 2017

2017-11-20 22:35 GMT+01:00 Eric Snow <ericsnowcurrently at gmail.com>:

I'm okay with that if we can't find another way. However, shouldn't we be able to statically initialize the raw allocator in PyRuntime, much as we were doing before in obmalloc.c? I have a rough PR up:

https://github.com/python/cpython/pull/4481 Also, I opened https://bugs.python.org/issue32096 for the regression. Thanks for bringing it up.

To statically initialize PyMemAllocatorEx fields, you need to export a lot of allocator functions. I would prefer to not do that.

static void* _PyMem_DebugRawMalloc(void ctx, size_t size); static void _PyMem_DebugRawCalloc(void ctx, size_t nelem, size_t elsize); static void _PyMem_DebugRawRealloc(void *ctx, void *ptr, size_t size); static void _PyMem_DebugRawFree(void *ctx, void *ptr);

static void* _PyMem_DebugMalloc(void ctx, size_t size); static void _PyMem_DebugCalloc(void ctx, size_t nelem, size_t elsize); static void _PyMem_DebugRealloc(void *ctx, void *ptr, size_t size); static void _PyMem_DebugFree(void *ctx, void *p);

static void* _PyObject_Malloc(void ctx, size_t size); static void _PyObject_Calloc(void *ctx, size_t nelem, size_t elsize); static void _PyObject_Free(void *ctx, void p); static void _PyObject_Realloc(void *ctx, void *ptr, size_t size);

The rules to choose the allocator to each domain are also complex depending if pymalloc is enabled, debug hooks are enabled by default, etc. The memory allocator is also linked to _PyMem_Debug which is not currently in Include/internals/ but Objects/obmalloc.c.

I understand that moving global variables to _PyRuntime helps to clarify how these variables are initialized and then finalized, but memory allocators are a complex corner case.

main(), Py_Main() and _PyRuntime_Initialize() now have to change temporary the allocators to make sure that their initialization and finalization use the same allocator.

I prefer to revert the change on memory allocators, and retry later to fix it, once other initializations issues are fixed ;-)

Victor

More information about the Python-Dev mailing list