[Python-Dev] C API for appending to arrays (original) (raw)

Mike Klaas mike.klaas at gmail.com
Wed Feb 4 01:25:54 CET 2009

On 2-Feb-09, at 9:21 AM, Hrvoje Niksic wrote:

It turns out that an even faster method of creating an array is by using the fromstring() method. fromstring() requires an actual string, not a buffer, so in C++ I created an std::vector with a contiguous array of doubles, passed that array to PyStringFromStringAndSize, and called array.fromstring with the resulting string. Despite all the unnecessary copying, the result was much faster than either of the previous versions.

Would it be possible for the array module to define a C interface for the most frequent operations on array objects, such as appending an item, and getting/setting an item? Failing that, could we at least make fromstring() accept an arbitrary read buffer, not just an actual string?

Do you need to append, or are you just looking to create/manipulate an
array with a bunch of c-float values? I find As{Write/Read}Buffer
sufficient for most of these tasks. I've included some example pyrex
code that populates a new array.array at c speed. (Note that you can
get the size of the resulting c array more easily than you are by
using PyObject_Length). Of course, this still leaves difficult
appending to an already-created array.

def calcW0(W1, colTotal): """ Calculate a W0 array from a W1 array.

 @param W1: array.array of doubles
 @param colTotal: value to which each column should sum

 @return W0 = [colTotal] * NA - W1
 """
 cdef int NA
 NA = len(W1)
 W0 = array('d', [colTotal]) * NA

 cdef double *cW1, *cW0
 cdef int i
 cdef Py_ssize_t dummy

 PyObject_AsReadBuffer(W1, <void**>&cW1, &dummy)
 PyObject_AsWriteBuffer(W0, <void**>&cW0, &dummy)

 for i from 0 <= i < NA:
     cW0[i] = cW0[i] - cW1[i]

 return W0

regards, -Mike

More information about the Python-Dev mailing list