[Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Thu Jun 20 01:01:49 CEST 2013

On 20 Jun 2013 02:03, "Victor Stinner" <victor.stinner at gmail.com> wrote:

2013/6/19 Kristján Valur Jónsson <kristjan at ccpgames.com>: > Oh, it should be public, in my opinion. Ok. And do you think that the PyMemMappingAllocator structure is complete, or that we should add something to be future-proof? At least, PyMemMappingAllocator is enough for pymalloc usage :-) Is PyMemMappingAllocator complete enough for your usage at CCP Games?

Can we go back to calling this the "Arena" allocator? Or at least "Mapped"? When I see "Mapping" in the context of Python I think of the container API, not a memory allocation API.

> We do exactly that when we embed python into UnrealEngine. We keep pythons internal PyObject_Mem allocator, but have it ask UnrealEngine for its arenas. That way, we can still keep track of python's memory usage from with the larger application, even if the granularity of memory is now on an "arena" level, rather than individual allocs. I hope that the PEP 445 is flexible enough to allow you to decide which functions are hooked and replaced, and which functions will be leaved unchanged. That's why I'm not in favor of the "Make PyMemMalloc() reuse PyMemRawMalloc() by default" alternative.

It's also why I'm in favour of the "domain" API rather than separate functions.

  1. In the initial iteration, just have the three basic domains (raw, interpreter, objects). Replacing allocators for third party libraries is the responsibility of embedding applications.

  2. In a later iteration, add "PyMem_AddDomain" and "PyMem_GetDomains" APIs so that extension modules can register new domains for wrapped libraries. Replacing allocators is still the responsibility of embedding applications, but there's a consistent API to do it.

(Alternatively, we could do both now)

And agreed PyMem_Raw* are the only new APIs that should be added to the stable ABI.

Cheers, Nick.

Victor

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20130620/73c9c52d/attachment.html>

More information about the Python-Dev mailing list