[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Jun 1 04:34:23 EDT 2017

2017-06-01 10:19 GMT+02:00 Antoine Pitrou <solipsis at pitrou.net>:

Yes, this is the same kind of reason the default page size is still 4KB on many platforms today, despite typical memory size having grown by a huge amount. Apart from the cost of fragmentation as you mentioned, another issue is when many small Python processes are running on a machine: a 2MB overhead per process can compound to large numbers if you have many (e.g. hundreds) such processes.

I would suggest we exert caution here. Small benchmarks generally have a nice memory behaviour: not only they do not allocate a lot of memory a, but often they will release it all at once after a single run. Perhaps some of those benchmarks would even be better off if we allocated 64MB up front and never released it :-)

By the way, the benchmark suite performance supports different ways to trace memory usage:

I wrote the code but I didn't try it yet :-) Maybe we should check the memory usage before deciding to change the arena size?

Victor

More information about the Python-Dev mailing list