[Python-Dev] Expose the array interface in Python 2.5? (original) (raw)

Travis Oliphant oliphant.travis at ieee.org
Fri Mar 17 20:32:47 CET 2006

Edward C. Jones wrote:

"Travis E. Oliphant" <oliphant.travis at ieee.org> wrote:

> It is very important for many people to get access to memory with some > description of how that memory should be interpreted as an array. > Several Python packages could benefit if Python had some notion of an > array interface that was at least supported in a duck-typing fashion. Which packages? Which people? Which constituencies?

I think I spell it out later. Do you really need to argue about whether or not an array interface is a useful thing? I thought we were beyond that and to the point of trying to figure out how to get the many groups to agree at least on a common interface.

"Travis E. Oliphant" <oliphant.travis at ieee.org> also wrote:

> My big quest is to get PIL, PyVox, WxPython, PyOpenGL, and so forth to > be able to use the same interface. Blessing the interface by > including it in the Python core would help. I'm also just wanting > people in py-dev to get the concept of an array interface on their > radar, as discussions of new bytes types emerges. I use PIL and numarray a lot. It would be nice if they used a common array format so I would not need to convert back and forth. But I survive quite well with the current arrangement.

We all "survive", but saying it is "quite well" is a bit optimistic as it means many very useful applications are harder to write than they really need to be.

Many other packages besides PIL and Numeric / numarray / Numpy are involved here: byte, struct, ctypes, SWIG, PyTables, Psyco, PyPy, Pyrex, etc. There are some major issues that need to be dealt with which I will

Sure they are involved, but I would argue the other ones you list care less about the multidimensional aspect of the array interface. (Actually PyTables just uses NumPy and so it should not be discussed as a "separate" package --- i.e. PyTables already tries to get along with NumPy as do many other packages...)

A data structure without an API and thorough documentation is useless. Any proposal needs to include them from the very start.

Again, I restate. The Numeric structure has been documented and has been around for a long time. I'm just trying to get this basic interface into Python as a very simple object. Let's not try to make it so complicated that no body can agree on what it should do. To be specific, I want to see a type object with almost none of the Type structure filled in with specific behavior.

I'm mainly interested in an array structure that other packages can rely on (and inherit from if they so choose).

Because the C-structure of the Numeric PyArrayObject (which NumPy also uses) is so widely known and used and documented for over 10 years, I argue it ought to form the foundation for this simple Python object.

We can argue about explicit multidimensional indexing behavior, but to hold hostage the introduction of a simple inheritable object to disagreements about those more complicated issues seems to be missing the mark.

How should Python interact with low level data? By "low level data" I mean data as seen by C, C++, and FORTRAN as well as linear arrays of bytes.

This is already known about in Numeric. That's what I'm saying. Numeric handles this well, let's just bring over this basic memory model for an array over to Python itself and not worry about the other TypeObject function-pointer tables until later.

Everybody I talked to at SciPy was very enthused about this concept. There is a large number of people who don't read Python-dev that I'm speaking for here.

What changes in Python would make the packages listed above easier to write and use? Easier enough to write that the package owners would be willing to give up control of part of their packages.

They don't have to give up control if we just introduce a simple memory model for an array.

Thanks for your comments,

-Travis

More information about the Python-Dev mailing list