[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Jun 1 05:03:04 EDT 2017

2017-06-01 10:41 GMT+02:00 Larry Hastings <larry at hastings.org>:

On 06/01/2017 01:19 AM, Antoine Pitrou wrote: If you'd like to go that way anyway, I would suggest 1MB as a starting point in 3.7.

I understand the desire for caution. But I was hoping maybe we could experiment with 4mb in trunk for a while? We could change it to 1mb--or even 256k--before beta 1 if we get anxious.

While I fail to explain why in depth, I would prefer to not touch the default arena size at this point.

We need more data, for example measure the memory usage on different workloads using different arena sizes.

It's really hard to tune a memory allocator for any use cases.

A simple enhancement would be to add an environment variable to change the arena size at Python startup. Example: PYTHONARENASIZE=1M. If you know that your application will allocate at least 2 GB, you may even want to try PYTHONARENASIZE=1G which is more likely to use a single large page... Such parameter cannot be used by default: it would make the default Python memory usage insane ;-)

Victor

More information about the Python-Dev mailing list