[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Jun 1 04:37:01 EDT 2017

2017-06-01 10:23 GMT+02:00 INADA Naoki <songofacandy at gmail.com>:

AFAIK, allocating arena doesn't eat real (physical) memory.

* On Windows, VirtualAlloc is used for arena. Real memory page is assigned when the page is used first time. * On Linux and some other *nix, anonymous mmap is used. Real page is assigned when first touch, like Windows.

Memory fragmentation is also a real problem in pymalloc. I don't think that pymalloc is designed to reduce the memory fragmentation.

I know one worst case: the Python parser which allocates small objects which will be freed when the parser completes, while other objects living longer are created. https://github.com/haypo/misc/blob/master/memory/python_memleak.py

In a perfect world, the parser should use a different memory allocator for that. But currently, the Python API doesn't offer this level of granularity.

Victor

More information about the Python-Dev mailing list