[Python-Dev] Is it kosher to use a buffer after release? (original) (raw)

Serhiy Storchaka storchaka at gmail.com
Sun May 10 20:46:16 CEST 2015

On 10.05.15 21:28, Larry Hastings wrote:

In Python's argument parsing code (convertsimple in Python/getargs.c), a couple of format units* accept "read-only bytes-like objects", aka read-only buffer objects. They call a helper function called convertbuffer() which uses the buffer protocol to extract a pointer to the memory.

Here's the relevant bit of code: static Pyssizet convertbuffer(PyObject *arg, void **p, char **errmsg) { Pybuffer view; ... if (getbuffer(arg, &view, errmsg) < 0) return -1; count = view.len; *p = view.buf; PyBufferRelease(&view); return count; }

getbuffer() uses the buffer protocol to fill in the "view" buffer. If it's successful, "view" is a valid buffer. We store the pointer to the buffer's memory in output parameter p. THEN WE RELEASE THE BUFFER. THEN WE RETURN TO THE CALLER. In case you missed the big helpful capital letters, we are returning a pointer given to us by PyObjectGetBuffer(), which we have already released by calling PyBufferRelease(). The buffer protocol documentation for bfreleasebuffer makes it sound like this pointer could easily be invalid after the release call finishes. Am I missing something, or is this code relying on an implementation detail it shouldn't--namely that you can continue using a pointer to some (most? all?) buffer memory even after releasing it?

You are missing following code:

 if (pb != NULL && pb->bf_releasebuffer != NULL) {
     *errmsg = "read-only bytes-like object";
     return -1;
 }

convertbuffer() is applicable only for types for which PyBuffer_Release() is no-op. That is why there are different format units for read-only buffers and for general buffers. That is why new buffer protocol was introduced.

More information about the Python-Dev mailing list