[Python-Dev] Buffer protocol for io.BytesIO? (original) (raw)

Guido van Rossum guido at python.org
Fri Sep 3 19:13:43 CEST 2010

On Fri, Sep 3, 2010 at 10:03 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

On Fri, 3 Sep 2010 09:32:22 -0700 Guido van Rossum <guido at python.org> wrote:

>> > It could not be resized, but it could be modified (same as what happens >> > with bytearrays today). Actually, the buffer itself would be writable, >> > and allow modifying the BytesIO contents. >> >> You may need to be careful with reads and writes while the buffer is >> exposed (e.g. throwing an exception until the buffer is released >> again). Perhaps the buffer accessor should be a context manager so it >> is easy to enforce prompt release? > > That's an interesting idea. I was planning to return a memoryview > object (in order to hide the intermediate object, and make it really > minimal), so perhaps the context protocol should be enabled on > memoryviews? > > (enter would be a no-op, and exit would release the internal > buffer and render it invalid, a bit like closing a file)

So far I'm  -0 on this. I'm not sure if it solves a real problem, and I think that if we were to add a way to define the scope of a buffer operation using a with-statement, it should work for all objects that support the buffer protocol (which IIUC means more than just memoryview). I'd be more enthusiastic about a separate context manager to wrap any such object. Most objects don't have a dedicated Python method to return a buffer, because they implement the buffer API implicitly at the C level, using stack-allocated Pybuffer structs. Therefore, there would be no point in a context manager for these objects (the buffer is released timely by the consuming function). It's only when creating a persistent Python-level wrapper for the Pybuffer struct (that is, a memoryview) that heap memory management issues could come into play. You are right that it's probably not a very important issue. Mutating an object's buffer from several threads isn't recommended anyway. As for resource consumption, what matters is when the original object (which owns the memory area) gets deallocated, and there's little chance that creating an additional memoryview makes a significant difference.

Thanks for the explanation. Still -0.

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list