[Numpy-discussion] Any Numeric or numarray users on this list? (original) (raw)

Fernando Perez fperez.net at gmail.com
Sun Jun 25 14:27:39 EDT 2006

On 5/30/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:

Please help the developers by responding to a few questions.

Sorry for not replying before, I wanted a more complete picture before answering.

1) Have you transitioned or started to transition to NumPy (i.e. import numpy)?

The day this email came in, I had just started to look into porting our major research code. I actually did the work 2 weeks ago, and it went remarkably well. It took a single (marathon) day, about 14 hours of solid work, to go through the old codebase and convert it. This project had a mix of legacy Fortran wrapped via f2py, hand-written C extensions using Numeric, a fair bit of weave.inline() and pure python. It uses matplotlib, PyX and Mayavi for various visualization tasks.

There are some 40k loc in the Fortran sources (2/3 of that auto-generated in python from Mathematica computations), and about 13k loc in the C and python sources.

This codebase is heavily unit-tested, which was critical for the port. For this kind of effort, unittests make an enormous difference, as they guide you directly to all the problematic spots. Without unittests, this kind of port would have been a nightmare, and I would have never known whether things were actually finished or not.

Most of my changes had to do with explicit uses of 'typecode=' which became dtype, and uses of .flat, which used to return a normal array and is now an iterator.

I haven't benchmarked things right away, because I expect the numpy-based code to take quite a hit. In this code, I've heavily abused arrays for very trivial 2 and 3-element arithmetic operations, but that means that I create literally millions of extremely small arrays. Even with Numeric, this overhead was already measurable, and I imagine it will get worse with numpy. But since this was silly anyway, and I need to use these little arrays as dictionary keys, instead of doing lots of tuple(array()) all the time, I'm using David Cooke's Vector as a template for a hand-written mini-array class that will do exactly what I need with as little overhead as possible.

If for any reason you do want to see actual benchmarks, I can try to run some with the codebases immediately before and after the Numeric->numpy change and report back.

2) Will you transition within the next 6 months? (if you answered No to #1)

That's it: by now we've moved all of our code and it doesn't really work with Numeric anymore, so we're committed :)

Again, many thanks for the major improvements that numpy brings!

Cheers,

f

More information about the NumPy-Discussion mailing list