[Python-Dev] pymalloc killer (original) (raw)

Martin v. Loewis martin@v.loewis.de
29 Mar 2002 15:58:18 +0100

Tim Peters <tim.one@comcast.net> writes:

So I have a variant of pymalloc that doesn't use magic cookies -- it knows "for sure", by maintaining a sorted (by address) contiguous vector of the base addresses of the (256KB each) "arenas" allocated by pymalloc.

I would not like to see such a binary search performed. Instead, if anything needs to be done, I'd be in favour of using a constant-time algorithm, even if it means that a littler more memory overhead is necessary.

I have the following idea: each chunk allocated by ought to have a pointer to its pool, immediately preceding the memory block. This will make an overhead of 4 bytes per allocation. Chunks allocated by the system allocator will have a null pointer preceding them.

To deal with alignment, the size classes would increase each by 4 bytes, so they would be spaced at 12, 20, 28, etc. bytes. With the 4 byte overallocation, each chunk would be 8-aligned, if the previous chunk in the same pool is.

This approach would allow to remove the requirement that each pool must be 4k-aligned.

To support the NULL pointer in a system-malloc'ed chunk, pymalloc would overallocate 8 bytes if it defers to system malloc, to preserve alignment.

What do you think?

Regards, Martin