[Python-Dev] Modify PyMem_Malloc to use pymalloc for performance (original) (raw)

M.-A. Lemburg mal at egenix.com
Thu Feb 4 05:17:51 EST 2016

On 03.02.2016 22:03, Victor Stinner wrote:

Hi,

There is an old discussion about the performance of PyMemMalloc() memory allocator. CPython is stressing a lot memory allocators. Last time I made statistics, it was for the PEP 454: "For example, the Python test suites calls malloc() , realloc() or free() 270,000 times per second in average." https://www.python.org/dev/peps/pep-0454/#log-calls-to-the-memory-allocator I proposed a simple change: modify PyMemMalloc() to use the pymalloc allocator which is faster for allocation smaller than 512 bytes, or fallback to malloc() (which is the current internal allocator of PyMemMalloc()). This tiny change makes Python up to 6% faster on some specific (macro) benchmarks, and it doesn't seem to make Python slower on any benchmark: http://bugs.python.org/issue26249#msg259445 Do you see any drawback of using pymalloc for PyMemMalloc()?

Yes: You cannot free memory allocated using pymalloc with the standard C lib free().

It would be better to go through the list of PyMem_*() calls in Python and replace them with PyObject_*() calls, where possible.

Does anyone recall the rationale to have two families to memory allocators?

The PyMem_*() APIs were needed to have a cross-platform malloc() implementation which returns standard C lib free()able memory, but also behaves well when passing 0 as size.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Experts (#1, Feb 04 2016)

Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/

::: We implement business ideas - efficiently in both time and costs :::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/

More information about the Python-Dev mailing list