[Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes (original) (raw)

Sturla Molden sturla.molden at gmail.com
Fri Jun 6 04🔞05 CEST 2014

On 05/06/14 22:51, Nathaniel Smith wrote:

This gets evaluated as:

tmp1 = a + b tmp2 = tmp1 + c result = tmp2 / c All these temporaries are very expensive. Suppose that a, b, c are arrays with N bytes each, and N is large. For simple arithmetic like this, then costs are dominated by memory access. Allocating an N byte array requires the kernel to clear the memory, which incurs N bytes of memory traffic.

It seems to be the case that a large portion of the run-time in Python code using NumPy can be spent in the kernel zeroing pages (which the kernel does for security reasons).

I think this can also be seen as a 'malloc problem'. It comes about because each new NumPy array starts with a fresh buffer allocated by malloc. Perhaps buffers can be reused?

Sturla

More information about the Python-Dev mailing list