[Python-Dev] Frame zombies (original) (raw)

Eyal Lotem eyal.lotem at gmail.com
Sun Jun 10 06:38:03 CEST 2007

The freelist currently serves as a good optimization of a special case of a recurring recursion. If the same code object (or one of the same size) is used for recursive calls repeatedly, the freelist will realloc-to-same-size (which probably has no serious cost) and thus the cost of allocating/deallocating frames was averted.

I think that in general, the complexity of a sophisticated and efficient aging mechanism is not worth it just to optimize recursive calls. The only question is whether it is truly a memory problem, if using, say, up-to 50 frames per code object?

Note that only recursions will have more than 1 frame attached.

How many recursions are used and then discarded? How slow is it to constantly malloc/free frames in a recursion?

My proposal will accelerate the following example:

def f(x, y): if 0 == x: return f(x-1, y) def g(x): if 0 == x: return g(x-1)

while True: f(100, 100) g(100)

The current implementation will work well with the following:

while True: f(100, 100)

But removing freelist altogether will not work well with any type of recursion.

Eyal

On 6/10/07, "Martin v. Löwis" <martin at v.loewis.de> wrote:

> Should I make a patch?

-1. This could consume quite a lot of memory, and I doubt that the speed improvement would be significant. Instead, I would check whether performance could be improved by just dropping the freelist. Looking at the code, I see that it performs a realloc (!) of the frame object if the one it found is too small. That surely must be expensive, and should be replaced with a free/malloc pair instead. I'd be curious to see whether malloc on today's systems is still so slow as to justify a free list. If it is, I would propose to create multiple free lists per size classes, e.g. for frames with 10, 20, 30, etc. variables, rather than having free lists per code object. Regards, Martin

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/eyal.lotem%40gmail.com

More information about the Python-Dev mailing list