[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)

Larry Hastings larry at hastings.org
Thu Jun 1 14:17:12 EDT 2017

On 06/01/2017 02:03 AM, Victor Stinner wrote:

2017-06-01 10:41 GMT+02:00 Larry Hastings <larry at hastings.org>:

On 06/01/2017 01:19 AM, Antoine Pitrou wrote:

If you'd like to go that way anyway, I would suggest 1MB as a starting point in 3.7.

I understand the desire for caution. But I was hoping maybe we could experiment with 4mb in trunk for a while? We could change it to 1mb--or even 256k--before beta 1 if we get anxious. While I fail to explain why in depth, I would prefer to not touch the default arena size at this point. We need more data, for example measure the memory usage on different workloads using different arena sizes.

I can't argue with collecting data at this point in the process. My thesis is simply "the correct value for this tunable parameter in 2001 is probably not the same value in 2017". I don't mind proceeding slowly or gathering more data or what have you for now. But I would like to see it change somehow between now and 3.7.0b1, because my sense is that we can get some performance for basically free by updating the value.

If ARENA_SIZE tracked Moore's Law, meaning that we doubled it every 18 months like clockwork, it'd currently be 2**10 times bigger: 256MB, and we'd be changing it to 512MB at the end of August.

(And yes, as a high school student I was once bitten by a radioactive optimizer, so these days when I'm near possible optimizations my spider-sense--uh, I mean, my optimization-sense--starts tingling.)

A simple enhancement would be to add an environment variable to change the arena size at Python startup. Example: PYTHONARENASIZE=1M.

Implementing this would slow down address_in_range which currently compiles in arena size. It'd be by a tiny amount, but this inline function gets called very very frequently. It's possible this wouldn't hurt performance, but my guess is it'd offset the gains we got from larger arenas, and the net result would be no faster or slightly slower.

//arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20170601/2f3e36ea/attachment-0001.html>

More information about the Python-Dev mailing list