[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)

Tim Peters tim.peters at gmail.com
Sun Jun 4 16🔞20 EDT 2017

[Larry Hastings <larry at hastings.org>]

... Yet CPython's memory consumption continues to grow. By the time a current "trunk" build of CPython reaches the REPL prompt it's already allocated 16 arenas.

I'd be surprised if that's true ;-) The first time new_arena() is called, it allocates space for a vector of 16 (INITIAL_ARENA_OBJECTS) arena_object structs. Those are tiny, and hold bookkeeping info for the actual arenas, none of which are allocated at first. new_arena() asks the system for memory for just one arena (and 15 of its 16 initial arena_object structs remain unused, awaiting future calls).

At the time, 16 wasn't picked because it was the minimum Python needed, but because it was the most the bulk of a variety of casual Python scripts needed to complete.

It's certainly gotten worse on a 64-bit box. Here on a 64-bit Win10 under Python 3.6.1 upon reaching the interactive prompt in a DOS box (set envar PYTHONMALLOCSTATS to get this kind of output whenever new_arena() is called):

arenas allocated total = 14

arenas reclaimed = 5

arenas highwater mark = 9

arenas allocated current = 9

9 arenas * 262144 bytes/arena = 2,359,296

bytes in allocated blocks = 2,166,656

bytes in available blocks = 140,104

0 unused pools * 4096 bytes = 0

bytes lost to pool headers = 27,648

bytes lost to quantization = 24,888

bytes lost to arena alignment = 0

Total = 2,359,296

So at most 9 arenas ("highwater mark") were ever simultaneously allocated..

Change the arena size to 64MB, and most programs would stop with the first arena allocated ;-)

More information about the Python-Dev mailing list