[Python-Dev] Re: Activating pymalloc (original) (raw)

Neil Schemenauer nas@python.ca
Mon, 18 Mar 2002 07:30:20 -0800

Vladimir Marangozov wrote:

I had a quick look. I'm not happy with it for two reasons:

- it removes all those useful comments in pymem.h and objimpl.h - the overall picture of the memory APIs in less clear than before (and there are no docs trying to clarify the issue)

It's not finished yet. I plan to straighten out the comments in the next version. The new APIs will have to be documented as well.

API Summary -----------

Raw malloc API: PyMem{MALLOC, REALLOC, FREE} PyMem{NEW, DEL} PyMem{Malloc, Realloc, Free} PyMem{New, Del} Object malloc API: PyObject{MALLOC, REALLOC, FREE} PyObject{Malloc, Realloc, Free} PyObject{New, NewVar, Del} PyObjectGC{New, NewVar, Del} Python's internal malloc API: PyCore{Malloc, Realloc, Free}

That doesn't solve the problem of broken extension modules (modules that mix PyMem_{NEW, DEL, MALLOC, FREE}, PyObject_{New, Del}, malloc and free or that call PyObject_* with the GIL).

We can do two things about this problem. First, we can say the hell with broken extension modules and keep the current API. Alternatively, we can expose a new object malloc API that is documented not to be thread safe and that may not use the system malloc() and free().

Personally, I don't care which approach we take but everyone must know which one we have decided on. If we decide to break the extension modules then we need to make sure users know about it when we upgrade (e.g. huge message in NEWS and on the release page, documentation on how to disable pymalloc if people have trouble).

If I get time today I will create another patch implementing the alternative approach.

Neil