[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)
Tim Peters tim.peters at gmail.com
Mon Jun 5 23:10:49 EDT 2017
- Previous message (by thread): [Python-Dev] The untuned tunable parameter ARENA_SIZE
- Next message (by thread): [Python-Dev] The untuned tunable parameter ARENA_SIZE
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
[Larry]
... Oh! I thought it also allocated the arenas themselves, in a loop. I thought I saw that somewhere. Happy to be proved wrong...
There is a loop in new_arena(), but it doesn't do what a casual
glance may assume it's doing ;-) It's actually looping over the
newly-allocated teensy arena descriptor structs, linking them in to a
freelist and recording that they're not (yet) associated with any
address space.
[Tim]
So at most 9 arenas ("highwater mark") were ever simultaneously allocated [by the time the REPL prompt appeared in a 64-bit 3.6.1]..
... though not completely off-base.
Yes, 9 is in the ballpark of 16.
I was hoping to spur a discussion of much higher level issues. I bet Larry was too ;-)
Actually I was hoping everyone would just tell me how right I was and thank me for my profound insights.
Thank you! It should be thought about again.
I think some increase of arena size should be a no-brainer, but I don't expect it to help a lot. For reasons already partly explained, I expect we'd get much better bang for the buck by increasing the pool size:
Roughly speaking, we bash into a slows-the-code pool boundary 64 times as frequently as an arena boundary. If the arena size increased, that ratio only gets worse.
On 64-bit boxes the bytes lost to pool headers increased, but the pool size did not. Thus we're guaranteed to "waste" a higher percentage of allocated bytes than we did on a 32-bit box.
The small object threshold also doubled. Generally (not always), the bytes lost to quantization increase the larger the size class. For example, for the largest size class now, on a 64-bit box we can only fit 7 512-byte objects into the 4096 - 48 = 4048 bytes that remain in a pool. So, in addition to the 48 pool header bytes, we also lose the 464 leftover bytes. So we've lost 512 of the 4096 bytes: 12.5%. That's certainly not typical, but in any case quantization losses as percentage of total bytes decrease the larger the pool.
Alas, I haven't thought of a plausibly practical way to replace Py_ADDRESS_IN_RANGE unless the pool size increases a whole frickin' lot :-(
- Previous message (by thread): [Python-Dev] The untuned tunable parameter ARENA_SIZE
- Next message (by thread): [Python-Dev] The untuned tunable parameter ARENA_SIZE
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]