[Python-Dev] Activating pymalloc (original) (raw)

Guido van Rossum guido@python.org
Fri, 15 Mar 2002 11:20:17 -0500

I wonder whether it's actually a good idea for PyMemNew (etc) to keep on calling malloc/free even when pymalloc is enabled. Anyone think they know a reason (other than theoretical purity ) for why pymalloc only targets the PyObject family? If Michael is right, the current distinction is indeed guaranteed to break just about every extension written for 1.5.2. It's also plain irritating that if you have, e.g., a gazillion tiny lists, only the list headers will enjoy pymalloc's speed & memory savings (the guts still get allocated via PyMemXXX).

Well, then you would have to provide yet another set of macros that don't require the GIL, for use by extensions that need to allocate memory while not holding the GIL. Of course, you could say, "then just use malloc/free", but the intention of pymem.h was that it would also be possible to provide a malloc/free substitute that was thread-safe -- unlike pymalloc. I don't know how realistic that is -- the only candidate would be GNU malloc but that would just replace the malloc/realloc/free entry points so wouldn't need any of the pymem.h-approved hackery.

--Guido van Rossum (home page: http://www.python.org/~guido/)