[Python-Dev] Activating pymalloc (original) (raw)

M.-A. Lemburg mal@lemburg.com
Fri, 15 Mar 2002 12:40:00 +0100

Tim Peters wrote:

[Michael Hudson] > I think one of the reasons lots of modules break is that back in the > 1.5.2 days, PyObjectDel didn't exist, and you were expected to use > PyMemDel. This now breaks. You mean this breaks if pymalloc is enabled, right? Else they're just more ways to spell "free()".

The problem is that if you have an extension which was written for 1.5.2 this extension will use PyMem_Del() for object deallocation rather than PyObject_Del(). This will break as soon as PyObject_Del() maps to something different than PyMem_Del() since the old code will continue to call PyMem_Del().

I wonder whether it's actually a good idea for PyMemNew (etc) to keep on calling malloc/free even when pymalloc is enabled. Anyone think they know a reason (other than theoretical purity ) for why pymalloc only targets the PyObject family?

Depends on how you define PyMem_* ...

If you are saying that these macros/functions are only intended to allocate memory blocks which Python will use, then it's probably OK to let pymalloc handle these as well.

If, for some reason, you intend to share the memory with some other process/DLL/whatever, then I'm not sure if this continues to work with pymalloc -- could be that the platform's malloc() uses flags/locks etc. to manage sharing. Just a thought, probably just FUD.

ALAS, the only way to find out, is to try it.

If Michael is right, the current distinction is indeed guaranteed to break just about every extension written for 1.5.2. It's also plain irritating that if you have, e.g., a gazillion tiny lists, only the list headers will enjoy pymalloc's speed & memory savings (the guts still get allocated via PyMemXXX).

Agreed.

-- Marc-Andre Lemburg CEO eGenix.com Software GmbH

Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/