[Numpy-discussion] Back to numexpr (original) (raw)

Tim Hochberg tim.hochberg at cox.net
Tue Jun 13 13:27:40 EDT 2006

David M. Cooke wrote:

On Tue, Jun 13, 2006 at 09:56:37AM -0700, Tim Hochberg wrote:

[SNIP]

All the above is checked in already :-) So I see. Oops!

- Support for both numpy and numarray (use the flag --force-numarray in setup.py). At first glance this looks like it doesn't make things to messy, so I'm in favor of incorporating this.

... although I had ripped this all out. I'd rather have a numpy-compatible numarray layer (at the C level, this means defining macros like PyArrayDATA) than different code for each. Okey dokey. I don't feel strongly about this either way other than I'd rather have one version of numexpr around rather than two almost identical versions. Whatever makes that work would makes me happy.

- Added a new benchmark for testing boolean expressions and strided/unaligned arrays: booleantiming.py Benchmarks are always good.

Haven't checked that in yet. Things that I want to address in the future: - Add tests on strided and unaligned data (currently only tested manually) Yep! Tests are good. - Add types for int16, int64 (in 32-bit platforms), float32, complex64 (simple prec.) I have some specific ideas about how this should be accomplished. Basically, I don't think we want to support every type in the same way, since this is going to make the case statement blow up to an enormous size. This may slow things down and at a minimum it will make things less comprehensible. I've been thinking how to generate the virtual machine programmatically, specifically I've been looking at vmgen from gforth again. I've got other half-formed ideas too (separate scalar machine for reductions?) that I'm working on too. But yes, the # of types does make things harder to redo :-) My thinking is that we only add casts for the extra types and do the computations at high precision. Thus adding two int16 numbers compiles to two OPCASTFfs followed by an OPADDFFF, and then a OPCASTfF. The details are left as an excercise to the reader ;-). So, adding int16, float32, complex64 should only require the addition of 6 casting opcodes plus appropriate modifications to the compiler. My thinking too. Yeah! Although I'm not in a hurry on this part.

I'm remembering now that the next item on my agenda was to work on supporting broadcasting. I don't exactly know how this is going to work, although I recall having something of a plan at some point. Perhaps the easiest way to start out is to just test the shapes of the input array for compatibility. If they're compatible and don't require broadcasting, proceed as now. If they are incompatible, raise a "ValueError: shape mismatch: objects cannot be broadcast to a single shape" as numpy does. If they are compatible, but require broadcasting, raise a NotImplementedError. This should be relatively easy and makes the numexpr considerably more congruent with numpy.

I'm hoping that, while working on that, my plan will pop back into my head ;-)

[SNIP]

Regards,

-tim

More information about the NumPy-Discussion mailing list