[Python-Dev] PEP: Adding data-type objects to Python (original) (raw)

Alexander Belopolsky alexander.belopolsky at gmail.com
Fri Nov 3 04:20:22 CET 2006

Paul Moore <p.f.moore gmail.com> writes:

Somewhat. My understanding is that the python-level buffer object is frowned upon as not good practice, and is scheduled for removal at some point (Py3K, quite possibly?) Hence, any code that uses buffer() feels like it "needs" to be replaced by something "more acceptable".

Python 2.x buffer object serves two distinct purposes. First, it is a "mutable string" object and this is definitely not going away being replaced by the bytes object. (Interestingly, this functionality is not exposed to python, but C extension modules can call PyBuffer_New(size) to create a buffer.) Second, it is a "view" into any object supporting buffer protocol. For a while this usage was indeed frowned upon because buffer objects held the pointer obtained from bf_get*buffer for too long causing memory errors in situations like this:

a = array('c', "x"*10) b = buffer(a, 5, 2) a.extend('x'*1000) str(b) 'xx'

This problem was fixed more than two years ago.

r35400 | nascheme | 2004-03-10

Make buffer objects based on mutable objects (like array) safe.

Even though it was suggested in the past that buffer object should be deprecated as unsafe, I don't remember seeing a call to deprecate the buffer protocol.

So although I understand the use you suggest, it's not compelling to me because I am left with the feeling that I wish I knew "the way to do it that didn't need the buffer object" (even though I realise intellectually that such a way may not exist).

As I explained in another post, I used buffer object as an example of an object that supports buffer protocol, but does not export type information in the form usable by numpy.

Here is another way to illustrate the problem:

a = numpy.array(array.array('H', [1,2,3])) b = numpy.array([1,2,3],dtype='H') a.dtype == b.dtype False

With the extended buffer protocol it will be possible for numpy.array(..) to realize that array.array('H', [1,2,3]) is a sequence of unsigned short integers and convert it accordingly. Currently numpy has to go through the sequence protocol to create a numpy.array from an array.array and loose the type information.

More information about the Python-Dev mailing list