[Python-Dev] An updated extended buffer PEP (original) (raw)

Carl Banks pythondev at aerojockey.com
Tue Mar 27 13:56:48 CEST 2007

Travis Oliphant wrote:

Travis Oliphant wrote:

Hi Carl and Greg,

Here is my updated PEP which incorporates several parts of the discussions we have been having. And here is the actual link: http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/doc/pepbuffer.txt

What's the purpose of void** segments in PyObject_GetBuffer? It seems like it's leftover from an older incarnation?

I'd hope after more recent discussion, we'll end up simplifying releasebuffer. It seems like it'd be a nightmare to keep track of what you've released.

Finally, the isptr thing. It's just not sufficient. Frankly, I'm having doubts whether it's a good idea to support multibuffer at all. Sure, it brings generality, but I'm thinking its too hard to explain and too hard to get one's head around, and will lead to lots of misunderstanding and bugginess. OTOH, it really doen't burden anyone except those who want to export multi-buffered arrays, and we only have one shot to do it. I just hope it doesn't confuse everyone so much that no one bothers.

Here's how I would update the isptr thing. I've changed "derefoff" to "subbufferoffsets" to describe it better.

typedef PyObject *(*getbufferproc)(PyObject *obj, void **buf, Py_ssize_t *len, int *writeable, char **format, int *ndims, Py_ssize_t **shape, Py_ssize_t **strides, Py_ssize_t **subbufferoffsets);

subbufferoffsets

Used to export information about multibuffer arrays. It is an address of a Py_ssize_t * variable that will be set to point at an array of Py_ssize_t of length *ndims.

[I don't even want to try a verbal description.]

To demonstrate how subbufferoffsets works, here is am example of a function that returns a pointer to an element of ANY N-dimensional array, single- or multi-buffered.

void* get_item_pointer(int ndim, void* buf, Py_ssize_t* strides,
                     Py_ssize_t* subarrayoffs, Py_ssize_t *indices) {
     char* pointer = (char*)buf;
     int i;
     for (i = 0; i < ndim; i++) {
         pointer += strides[i]*indices[i];
         if (subarraysoffs[i] >= 0) {
             pointer = *(char**)pointer + subarraysoffs[i];
         }
     }
     return (void*)pointer;
 }

For single buffers, subbufferoffsets is negative in every dimension and it reduces to normal single-buffer indexing. For multi-buffers, subbufferoffsets indicates when to dereference the pointer and switch to the new buffer, and gives the offset into the buffer to start at. In most cases, the subbufferoffset would be zero (indicating it should start at the beginning of the new buffer), but can be a positive number if the following dimension has been sliced, and thus the 0th entry in that dimension would not be at the beginning of the new buffer.

Other than that, looks good. :)

Carl Banks

More information about the Python-Dev mailing list