[Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes (original) (raw)

Terry Reedy tjreedy at udel.edu
Fri Jun 6 00:57:32 CEST 2014

On 6/5/2014 4:51 PM, Nathaniel Smith wrote:

In fact, AFAICT it's 100% correct for libraries being called by regular python code (which is why I'm able to quote benchmarks at you :-)). The bytecode eval loop always holds a reference to all operands, and then immediately DECREFs them after the operation completes. If one of our arguments has no other references besides this one, then we can be sure that it is a dead obj walking, and steal its corpse.

But this has a fatal flaw: people are unreasonable creatures, and sometimes they call Python libraries without going through ceval.c :-(. It's legal for random C code to hold an array object with a single reference count, and then call PyNumberAdd on it, and then expect the original array object to still be valid. But who writes code like that in practice? Well, Cython does. So, this is no-go.

I understand that a lot of numpy/scipy code is compiled with Cython, so you really want the optimization to continue working when so compiled. Is there a simple change to Cython that would work, perhaps in coordination with a change to numpy? Is so, you could get the result before 3.5 comes out.

I realized that there are other compilers than Cython and non-numpy code that could benefit, so that a more generic solution would also be good. In particular

Here's the idea. Take an innocuous expression like:

result = (a + b + c) / c

This gets evaluated as:

tmp1 = a + b
tmp2 = tmp1 + c
result = tmp2 / c

... There's an obvious missed optimization in this code, though, which is that it keeps allocating new temporaries and throwing away old ones. It would be better to just allocate a temporary once and re-use it: tmp1 = a + b tmp1 += c tmp1 /= c result = tmp1

Could this transformation be done in the ast? And would that help?

A prolonged discussion might be better on python-ideas. See what others say.

-- Terry Jan Reedy

More information about the Python-Dev mailing list