[Python-Dev] PEP-298: buffer interface widening too much? (original) (raw)

Todd Miller jmiller@stsci.edu
Sat, 14 Dec 2002 13:45:55 -0500

I've been following the PEP-298 discussion a little, because I fear eventually IAGNI. But when I look at the proposed changes, I can't help but get the feeling that the "too wide" buffer interface just got wider. By "too wide", I am referring to the extraneous segmentation and character buffer aspects of the existing interface. Adding three more calls for the locking problem bugs me. Doesn't anything ever get simpler?

Looking at the proposed changes, the only thing that really seems needed for the bufferprocs is the release call and a flag bit. Assuming the release call is not implemented, the acquired buffer is never locked or unlocked, and we're no worse off than we are now: it's the BO's problem. Assuming the release call is present, the acquired buffer should always be locked and later released. The existing wrapper functions (plus one for release) can hide/implement this, making it straight forward to provide correct extensions: just implement a locking extension. It seems to me that the existing (3!) calls for acquiring the buffer pointer can do just fine. Except...

In the case of existing extensions, there will certainly be no call to the release function. So for an extension which is slow to update to the revised protocol, there are negative consequences: buffers which are never unlocked. But so what? These consequences seem less dire (if more probable) than the existing consequences of "moving buffers." It would be a bigger problem if PEP-298 wound up fiddling with reference counts. In that case, un-released buffers would also become memory leaks. But is it really necessary to mess with the reference counts? I'm having a hard time imagining a situation in which the reference count for a buffer goes to zero while it is locked which is not a bug somewhere else. What am I missing?

Another possibility is to implement a "release buffer pointer" wrapper function which does nothing for a Python release 2.3 and change nothing else. Let extensions which use buffers adopt usage of the wrapper and the locking protocol, with no immediate benefit. Then implement locking for real in Python 2.4. That way, the number of non-conforming extensions can be minimized, but the buffer API can be kept simple.

So, in the end, what I'm wondering is whether there's a simpler middle ground between dropping the PEP and implementing it as it stands now.
Are there compromises to be found between guaranteeing correctness, backwards compatability, and growing complexity? Is there another trade between backwards compatability and complexity which is tolerable for the short term with both simplicity and correctness available in the longer term?

Todd