[C++-sig] create boost array from a Numpy PyArray without copying data? (original) (raw)

Sebastian Walter walter at mathematik.hu-berlin.de
Thu Jan 22 11:51:20 CET 2009

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Ravi schrieb:

On Wednesday 21 January 2009 03:40:00 Sebastian Walter wrote:

Apparently I have to increase the refcount manually when I return a raw PyObject. So I wrote getter functions like that:

PyObject* getrhs(){ return incref((PyObject *) arraymyrhs); } I'm not sure that this is the way it should be done. But it solved the memory corruption problem. In this case, you are essentially creating a memory leak. The returned array will never be deleted by python since its reference count will never go to zero. I'm sorry, the above code is not really self contained and now I'm not sure that we are talking about the same thing. I do not intend to EVER free the memory of *x!! So I don't have to worry about “make sure the memory doesn’t get deallocated before the NumPy array disappears.” because *x is global!

So here is the code that I actually wanted to post:

#define PY_ARRAY_UNIQUE_SYMBOL PyArrayHandle #include #include <boost/python.hpp> #include <numpy/noprefix.h>

using namespace std; using namespace boost; using namespace boost::python; using namespace boost::python::numeric;

double x[] = {1.,2.,3.}; int N = 3;

class A{ public: PyArrayObject *my_array;

A(){
    my_array  = (PyArrayObject*)PyArray_SimpleNewFromData(1, &N,

PyArray_DOUBLE, x); }

PyObject* get_my_array(){
    return incref((PyObject*) my_array);
}

};

BOOST_PYTHON_MODULE(_test) { import_array(); array::set_module_and_type("numpy", "ndarray");

class_<A>("A", init<>())
.add_property("my_array", &A::get_my_array)
;

}

def test_my_array(): a = A() print a.my_array print a.my_array

If I don't put the incref I get an *** glibc detected *** python: malloc(): memory corruption (fast): 0x0817b6a1 ***

Please see http://blog.enthought.com/?p=62 for information on fixing this issue. The code I pointed out earlier encapsulates this using a custom deleter in C++ which can be used independently of ublas.

I cite from that blog: """The suggestions have always been something similar to “make sure the memory doesn’t get deallocated before the NumPy array disappears.” This is nice advice, but not generally helpful as it basically just tells you to create a memory leak.

All that NumPy does internally is to un-set a flag on the array object indicating that the array doesn’t own its memory pointer and so NumPy won’t free the memory when the last reference to the array disappears."""

So that means: In my above case I know that *x won't be deleted because it is global, right? Why exactly does the *** glibc detected *** python: malloc(): memory corruption (fast): 0x0817b6a1 *** occur then?

And is there a way to get the refcount of an object within Python? I'd like to test if the refcount is correct.

regards, Sebastian -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFJeE+o9PBA5IG0h0ARAjeQAKCR+HQQ2XjiqLAO01ZwWRqIC5KUcgCgl+ne b80vmGWmi6jLLuxeOyjOt0w= =mmAr -----END PGP SIGNATURE-----

More information about the Cplusplus-sig mailing list