| msg220721 - (view) |
Author: Armin Rigo (arigo) *  |
Date: 2014-06-16 14:03 |
|
| Following the documentation at https://docs.python.org/3/c-api/buffer.html, if we write a custom object in C with a getbufferproc that simply returns PyBuffer_FillInfo(...), then it works fine up to Python 3.2 but segfaults in Python 3.3 depending on what we do with it. The segfault is caused by memoryobject.c:last_dim_is_contiguous(), which reads from the "strides" array without checking that it is not NULL. Attached the simplest example of C extension module I could make. When executing this: >>> import xy >>> m=memoryview(bytearray(b"abcdef")) >>> m[:5] = xy.gm ...it segfaults in Python 3.3. (I'm told it works fine in 3.2 and segfaults in 3.4 as well, but didn't confirm it.) |
|
|
|
| msg220727 - (view) |
Author: Stefan Krah (skrah) * |
Date: 2014-06-16 14:29 |
|
| The flags from mb_getbuf() have to be passed to PyBuffer_FillInfo(): return PyBuffer_FillInfo(view, (PyObject *)self, internal, 5, /*readonly=*/0, flags); The flags contain the actual request that the consumer sends to the buffer provider, so they cannot be hardcoded. In this example, memoryview sends the PyBUF_FULL request to mb_getbuf(). If the request is successful, it can assume non-NULL strides. Should the documentation be improved? |
|
|
|
| msg220730 - (view) |
Author: Armin Rigo (arigo) * |
Date: 2014-06-16 15:20 |
|
| Ah! Thanks. I systematically missed the "flags" arguments passed in the getbufferproc function. (I thought I had to tell PyBuffer_FillInfo() what kind of format we actually provide, but it seems that the interface is the other way around. I don't see how I would implement a getbufferproc that forces a specific format, but I don't need that, so it's fine.) |
|
|
|
| msg220741 - (view) |
Author: Stefan Krah (skrah) * |
Date: 2014-06-16 17:49 |
|
| Yes, PyBuffer_FillInfo() is a bit confusing: The canonical usage *looks* as if "flags" were somehow used in the Py_buffer struct to mark a buffer as having certain capabilities: Modules/_io/bufferedio.c: ------------------------- if (PyBuffer_FillInfo(&buf, NULL, start, len, 0, PyBUF_CONTIG) == -1) return -1; memobj = PyMemoryView_FromBuffer(&buf); What really goes on since Python 3.3 is that memobj is a memoryview with (shape,strides,format), since missing Py_buffer values are reconstructed. So all memoryviews -- when acting as an exporter as in this example -- have PyBUF_FULL capabilities. To make that clear, we should actually change the PyBUF_CONTIG in the above example to PyBUF_FULL. Then the mental model is: PyBuffer_FillInfo(..., PyBUF_FULL) ^ | "Request" full buffer to export. memobj ^ |
Request anything from PyBUF_SIMPLE to PyBUF_FULL. Note |
that e.g. with PyBUF_SIMPLE the consumer's shape and |
strides will be NULL! Consumer And that model is consistent with the usage of PyBuffer_FillInfo() inside a getbufferproc, where "flags" of course can be anything. |
| msg220742 - (view) |
Author: Stefan Krah (skrah) * |
Date: 2014-06-16 18:01 |
|
| I think it's worth adding a note to the docs about passing the flags unmodified inside a getbufferproc. |
|
|
|
| msg221908 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2014-06-29 22:16 |
|
| New changeset 49cdb04bfcc6 by Stefan Krah in branch '3.4': Issue #21778: Clarify use of flags if PyBuffer_FillInfo() is used inside a http://hg.python.org/cpython/rev/49cdb04bfcc6 |
|
|
|