[Python-Dev] Allocation of shape and strides fields in Py_buffer (original) (raw)

Travis Oliphant oliphant.travis at ieee.org
Wed Dec 10 17:34:09 CET 2008

Nick Coghlan wrote:

Antoine Pitrou wrote:

In all honesty, I admit I am annoyed by all the problems with the buffer API / memoryview object, many of which are caused by its utterly bizarre design (and the fact that the design team went missing in action after imposing such a bizarre and complex design on us), and I'm reluctant to add yet another level of byzantine complexity in order to solve those problems. It explains I may sound a bit angry at times :-)

If we really need to change things a lot to make them work, we should re-work the buffer API from the ground up, make the Pybuffer struct a true PyObject (that is, a true variable-length object so as to solve the shape and strides allocation issue) and merge it with the current memoryview implementation. It would make things both more simpler and more flexible. I don't see anything wrong with the PEP 3118 protocol. It does exactly what it is designed to do: allow the number crunching crowd to share large datasets between different libraries without copying things around in memory. Yes, the protocol is complicated, but that is because it is trying to handle a complicated problem. The memoryview implementation on the other hand is pretty broken. I do have a theory on how it ended up in such an unusable state, but I'm not particularly inclined to share it - this kind of thing can happen sometimes, and the important question now is how we fix it.

Thank you Nick. This is a correct assessment of the situation. I'd like to help improve memoryview as I can. It does need thought about what you want memoryview to be.

I wanted memoryview to be able to be sliced and diced (much like NumPy arrays). But, I only was able to get around to implementing the (simple view of Py_buffer struct).

-Travis

More information about the Python-Dev mailing list