[Python-Dev] Modify PyMem_Malloc to use pymalloc for performance (original) (raw)

Victor Stinner victor.stinner at gmail.com
Thu Feb 4 08:25:05 EST 2016

Thanks for your feedback, you are asking good questions :-)

2016-02-04 13:54 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:

There are 536 calls to the functions PyMemMalloc(), PyMemRealloc() and PyMemFree().

I would prefer to modify a single place having to replace 536 calls :-/ You have a point there, but I don't think it'll work out that easily, since we are using such calls to e.g. pass dynamically allocated buffers to code in extensions (which then have to free the buffers again).

Ah, interesting. But I'm not sure that we delegate the responsability of freeing the memory to external libraries. Usually, it's more the opposite: a library gives us an allocated memory block, and we have to free it. No?

I checked if we call directly malloc() to pass the buffer to a library, but I failed to find such case.

Again, in debug mode, calling free() on a memory block allocated by PyMem_Malloc() will likely crash. Since we run the Python test suite with a Python compiled in debug mode, we would already have detected such bug, no?

See also my old issue http://bugs.python.org/issue18203 which replaced almost all direct calls to malloc() with PyMem_Malloc() or PyMem_RawMalloc().

Good question. I guess developers simply thought of PyObjectMalloc() being for PyObjects,

Yeah, I also understood that, but in practice, it looks like PyMem_Malloc() is slower than so using it makes the code less efficient than it can be.

Instead of teaching developers that well, in fact, PyObject_Malloc() is unrelated to object programming, I think that it's simpler to modify PyMem_Malloc() to reuse pymalloc ;-)

Victor

More information about the Python-Dev mailing list