[Numpy-discussion] Back to numexpr (original) (raw)

David M. Cooke cookedm at physics.mcmaster.ca
Tue Jun 13 15:44:13 EDT 2006

On Tue, 13 Jun 2006 21:30:41 +0200 Francesc Altet <faltet at carabos.com> wrote:

A Dimarts 13 Juny 2006 20:46, Tim Hochberg va escriure: > >Uh, I'm afraid that yes. In PyTables, int64, while being a bit bizarre > >for some users (specially in 32-bit platforms), is a type with the same > >rights than the others and we would like to give support for it in > >numexpr. In > > fact, Ivan Vilata already has implemented this suport in our local copy > > of numexpr, so perhaps (I say perhaps because we are in the middle of a > > big project now and are a bit scarce of time resources) we can provide > > the patch against the latest version of David for your consideration. > > With this we can solve the problem with int64 support in 32-bit > > platforms (although addmittedly, the VM gets a bit more complicated, I > > really think that this is worth the effort) > > In addition to complexity, I worry that we'll overflow the code cache at > some point and slow everything down. To be honest I have no idea at what > point that is likely to happen, but I know they worry about it with the > Python interpreter mainloop.

That's true. I didn't think about this :-/ > Also, it becomes much, much slower to > compile past a certain number of case statements under VC7, not sure > why. That's mostly my problem though. No, this is a general problem (I'd say much more in GCC, because the optimizer runs so slooooow). However, this should only affect to poor developers, not users and besides, we should find a solution for int64 in 32-bit platforms.

If I switch to vmgen, it can easily make two versions of the code: one using a case statement, and another direct-threaded version for GCC (which supports taking the address of a label, and doing a 'goto' to a variable). Won't solve the I-cache problem, though. And there's always subroutine threading (each opcode is a function, and the program is a list of function pointers).

We won't know until we try :)

Or perhaps David can come with a better solution (vmgen from gforth? no idea what this is, but the name sounds sexy;-)

The docs for it are at http://www.complang.tuwien.ac.at/anton/vmgen/html-docs/

-- |>|/|< /--------------------------------------------------------------------------
|David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca

More information about the NumPy-Discussion mailing list