[Python-Dev] [ python-Patches-876206 ] scary frame speed hacks (original) (raw)

Christian Tismer tismer at stackless.com
Tue Mar 2 19:34:48 EST 2004

Tim Peters wrote:

[Skip]

Why not a list of pre-made frames for each code block (default length 1)? Or, IOW, per-code-block frame freelists (plural). Recursive generators have gotten quite popular in my code .

This is exactly what I was going to propose. A freelist per code object would be almost as efficient as the current freelist.

But the drawback... (well, Tim is first here, too :)

BTW, if a gazillion distinct functions run when starting up a large app, do we hang on to the memory for their gazillion distinct frames forever? Recycling from a common frame pool has memory benefits in cases other than just recursion. Experiment: run test.py from a Zope 2 or Zope 3 checkout, and look at highwater memory consumption with and without the patch.

What I used in Stackless was an array of cached objects which is indexed by object size, only for small sizes of course. When the total number of cached zombies reaches some watermark, I do a total clear of all the cache.

My proposal for this frame caching is as follows: Keep a small array of cached pre-initialized frames, but don't index it by size, but a very simple hash function. Allocation is then dividing the address (==id()) of the code object by some prime number, index the cache and see if there is the identical code object in the frame. If it is, take it, otherwise malloc a new frame. On deallocation, insert the frame into its proper slot. Test the total number of cached frames, and if it reaches the watermark, clear the whole cache.

cheers - chris

Christian Tismer :^) <mailto:tismer at stackless.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : Starship http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

More information about the Python-Dev mailing list