[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)

Larry Hastings larry at hastings.org
Thu Jun 1 23:21:12 EDT 2017

On 06/01/2017 02:50 AM, Antoine Pitrou wrote:

Another possible strategy is: allocate several arenas at once (using a larger mmap() call), and use MADVDONTNEED to relinquish individual arenas.

Thus adding a fourth layer of abstraction over memory we get from the OS?

block -> pool -> arena -> "multi-arena" -> OS

Y'know, this might actually make things faster. These multi-arenas could be the dynamically growing thing Victor wants to try. We allocate 16mb, then carve it up into arenas (however big those are), then next time allocate 32mb or what have you. Since the arenas remain a fixed size, we don't make the frequently-used code path (address_in_range) any slower. The code to deal with the multi-arenas would add a little complexity--to an admittedly already complex allocator implementation, but then what allocator isn't complex internally?--but it'd be an infrequent code path and I bet it'd be an improvement over simply calling malloc / mmap / VirtualAlloc. What do you think, Victor?

And to think I started this reply ironically,

//arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20170601/46762760/attachment.html>

More information about the Python-Dev mailing list