[Python-Dev] cpython: Issue #3329: Add new APIs to customize memory allocators (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Sat Jun 15 16:38:39 CEST 2013

On Sun, 16 Jun 2013 00:12:02 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote:

On 15 June 2013 22:41, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Sat, 15 Jun 2013 22:22:33 +1000 > Nick Coghlan <ncoghlan at gmail.com> wrote: >> For >> custom allocators, it's useful to be able to ensure you can bypass >> CPython's small object allocator, rather than having to rely on it >> being bypassed for allocations above a certain size. > > Which custom allocators?

Those used by companies like Dropbox to speed up frequent allocations (look up their PyCon 2011 keynote). If we don't provide suitable APIs that we can still hook in debug mode, they'll bypass our infrastructure completely and we'll miss significant memory accesses.

I don't understand the concern. People can ignore the Python allocators, and then use their own debugging infrastructure. This is what happens everytime you want to use your own infrastructure instead of a 3rd party-provided one.

> And I'm not even sure what "faux simplicity" you are talking about. > What is the supposed complexity that this API is supposed to address?

The fact that there is more to the world than x86/x8664 and the very simplistic C memory model.

Then I think it needs a PEP to properly address it and explain it to everyone.

Moreover, I think you are conflating two issues: the ability to add memory allocation hooks (for tracing/debugging purposes), and the adaptation to "non-traditional" memory models (whatever that means). Those concerns don't necessarily come together.

Regards

Antoine.

More information about the Python-Dev mailing list