[Python-Dev] Re: Activating pymalloc (original) (raw)

Tim Peters tim.one@comcast.net
Tue, 19 Mar 2002 23:39:24 -0500

[Andrew MacIntyre]

It works just fine when pymalloc is configured normally (~2% speedup). [when running] pystone.py, MAL's PyBench - nothing real world though...

Ah. PyBench will be more interesting when the small object threshold is boosted -- parts of it use oodles of tiny dicts, and pymalloc can't handle them now.

[when redirecting all (I think) mallocs to go thru pymalloc]

testlongexp was the pathological case.

It's a pathological case on some boxes without pymalloc. IIRC, it takes so long to finish on some flavor(s) of Mac that nobody has waited for it to complete (we're talking over half an hour here).

I just tried it on my (Windows) box, after redefining PyMem_{MALLOC, REALLOC, FREE} to resolve to PyMalloc{Malloc, Realloc, Free}. Ahem. That was total disaster, since the call to PyMem_MALLOC at the end of _PyMalloc_Malloc became unbounded recursion. Doh!

I then further bashed obmalloc.c to call malloc/realloc/free directly. test_longexp sped up significantly then, taking about 3 seconds before this fiddling and about 2 seconds after.

So best guess is that test_longexp + pymalloc-all-the-time tickles some pathology in your platform's malloc/realloc/free (or did). Do you have a lot of RAM? test_longexp is a memory hog regardless.