[Numpy-discussion] dealing with large arrays (original) (raw)
Ivan Vilata i Balaguer ivilata at carabos.com
Wed Jun 14 05:42:31 EDT 2006
- Previous message (by thread): [Numpy-discussion] dealing with large arrays
- Next message (by thread): [Numpy-discussion] finding connected areas?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
En/na Mathew Yeates ha escrit::
I typically deal with very large arrays that don't fit in memory. How does Numpy handle this? In Matlab I can use memory mapping but I would prefer caching as is done in The Gimp.
Hi Mathew. If you are in the need of storing large arrays on disk, you may have a look at Pytables_. It will save you some headaches with the on-disk representation of your arrays (it uses the self-describing HDF5 format), it allows you to load specific slices of arrays, and it provides caching of data. The latest versions also support numpy.
Hope that helps,
.. _PyTables: http://www.pytables.org/
:: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ Cárabos Coop. V. V V Enjoy Data ""
-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 307 bytes Desc: OpenPGP digital signature URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20060614/a39ff30e/attachment-0001.sig>
- Previous message (by thread): [Numpy-discussion] dealing with large arrays
- Next message (by thread): [Numpy-discussion] finding connected areas?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]