[Python-Dev] Extended Buffer Interface/Protocol (original) (raw)

Travis Oliphant oliphant at ee.byu.edu
Tue Mar 27 01:48:53 CEST 2007

Greg Ewing wrote:

But since the NumPy object has to know about the provider, it can simply pass the release call on to it if appropriate. I don't see how this case necessitates making the release call on a different object.

I'm -1 on involving any other objects or returning object references from the buffer interface, unless someone can come up with a use case which actually requires this (as opposed to it just being something which might be "nice to have"). The buffer interface should be Blazingly Fast(tm), and messing with PyObject*s is not the way to get that.

The current proposal would be fast but would be more flexible for objects that don't have a memory representation that can be shared unless they create their own "sharing object" that perhaps copies the data into a contiguous chunk first. Objects which have memory which can be shared perfectly through the interface would simply pass themselves as the return value (after incrementing their "extant buffers" count by one).

Seems to me the lock should apply to everything returned by getbuffer. If the requestor is going to iterate over the data, and there are multiple dimensions, surely it's going to want to refer to the shape and stride info at regular intervals while it's doing that. Requiring it to make its own copy would be a burden.

There are two use cases that seem to be under discussion.

  1. When you want to apply an algorithm to an arbitrary object that exposes the buffer interface

  2. When you want to create an object that shares memory with another object exposing the buffer interface.

These two use cases have slightly different needs. What I want to avoid is forcing the exporting object to be unable to change its shape and strides just because an object is using the memory for use case #2.

I think the solution that states the shape and strides information are only guaranteed valid until the GIL is released is sufficent.

Alternatively, one could release the shape and strides and format separately from the memory with a flag as a second argument to releasebuffer.

-Travis

-- Greg

More information about the Python-Dev mailing list