[Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Jun 20 07:37:42 CEST 2013

Le jeudi 20 juin 2013, Nick Coghlan a écrit :

> Is PyMemMappingAllocator complete enough for your usage at CCP Games? Can we go back to calling this the "Arena" allocator? Or at least "Mapped"? When I see "Mapping" in the context of Python I think of the container API, not a memory allocation API. This function is written to be able to use mmap() and VirtualAlloc(). There is no Python function to use directly this allocator yet, but I chose "memory mapping" name because it is very different than the heap and it may be useful for other functions than pymalloc.

If I change the name, it would be called PyObject_SetArenaAllocator() with a PyObjectArenaAllocator structure. I'm not sure that PyMemMappingAllocator API is future-proof, so I'm fine to call it "arena" again.

> I hope that the PEP 445 is flexible enough to allow you to decide > which functions are hooked and replaced, and which functions will be > leaved unchanged. That's why I'm not in favor of the "Make > PyMemMalloc() reuse PyMemRawMalloc() by default" alternative.

It's also why I'm in favour of the "domain" API rather than separate functions. 1. In the initial iteration, just have the three basic domains (raw, interpreter, objects). Replacing allocators for third party libraries is the responsibility of embedding applications. 2. In a later iteration, add "PyMemAddDomain" and "PyMemGetDomains" APIs so that extension modules can register new domains for wrapped libraries. Replacing allocators is still the responsibility of embedding applications, but there's a consistent API to do it. (Alternatively, we could do both now) How would you use an allocator of a new domain? PyMemBlockAllocator structure is not convinient, and if Py_GetAllocator() only once, you may loose a hook installed later.

Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20130620/b3b40587/attachment-0001.html>

More information about the Python-Dev mailing list