[Numpy-discussion] A little about XND (original) (raw)

Travis Oliphant teoliphant at gmail.com
Sun Jun 17 02:51:03 EDT 2018

Hi everyone,

I'm glad I'm able to contribute back to this discussion thread. I wanted to post a quick message to this group to make sure there is no mis-information about XND which has finally reached the point where it can be experimented with (http://xnd.io) and commented on.

XND came out of thoughts and conversations we had at Continuum (now Anaconda) when thinking about cross-language array computing and how to enable improved features for high-level users in many languages (including Python, R, Ruby, Node, Scala, Rust, Go, etc.). Technically there are three projects that make up XND (thus the name Plures for the Github organization). All of these projects have a C-library and then a high-level interface (right now we only have resources to develop the Python interface but would love to see support for other languages). xnd (libxnd) is the typed container. ndtypes (libndtypes) is the (datashape-like) type system with a grammar, parser, and type matcher. gumath (libgumath) are generalized ufuncs which represent the entire function system on xnd.

We will be talking more about XND in the coming months and years, but for the purposes of this list, I wanted to make it clear that

  1. XND is not trying to replace NumPy. XND is a low-level library and intended to be such. It would be most welcome if someday NumPy uses XND. We understand this may be a while and certainly not before NumPy 2.0 or 3.0.

  2. Our initial target users are Numba, pandas, Dask, xarray, and other higher-level objects at the moment. We are eagerly searching for integration opportunities to connect more developers (or advanced users) to xnd before making more progress.

  3. We do discuss array-like things in the public channels. NumPy users and developers are welcome in those channels. Everything is done in public including the weekly meeting which anyone can attend:

Weekly meeting: meet.google.com/heo-fmow-omz

Live discussions: https://gitter.im/Plures/xnd for the libraries themselves https://gitter.im/Plures/xnd-ml for integrations.

Issues and PRs: https://github.com/plures --- under the various projects.

  1. We are thinking about adding a custom-dtype to NumPy that uses xnd and would be happy for anyone's help on that project.

  2. We are in the early stages of exploring a high-level array interface (using the ideas of MoA and the Psi Calculus with Lenore Mullen who worked on APL years ago). Likely the first place this will see some initial progress is in an ND Sparse array that uses XND.

We welcome participation and input from all. Stefan Krah has written the majority of the code and so we tend to respect his point of view. Pearu Peterson (of f2py and SciPy fame) has made some useful contributions recently.

Stefan and I have been talking roughly weekly for a couple of years and so some of the problems currently there, I am certainly responsible for.

Two of our immediate goals are to work with the Numba team to get support for ndtypes in Numba and allow Numba to use libgumath in no-python mode.

I look forward to continuing the conversation with any of you who want to participate. Perhaps some of us can meet up during NumPy sprints to discuss more.

XND is also currently looking for funding and time from interested parties to continue its development.

-Travis -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180617/44d1f72b/attachment.html>

More information about the NumPy-Discussion mailing list