[Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API (original) (raw)

Marten van Kerkwijk m.h.vankerkwijk at gmail.com
Sun Jun 3 20:45:56 EDT 2018

This in certainly true in general, but given the complete flexibility of array_function there's no way we can make every check convenient. The best we can do is make it easy to handle the common cases, where the argument position does not matter.

I think those cases may not be as common as you think - most functions are not like concatenate & co... Indeed, it might be good to add some other examples to the NEP. Looing at the list of functions which do not work with Quantity currently: Maybe np.dot, np.choose, and np.vectorize?

Possibly, a solution would rely on the same structure as used for the "dance". But as a general point, I don't see the advantage of passing types rather than arguments - less information for no benefit.

Maybe this is premature optimization, but there will certainly be fewer unique types than arguments to check for types. I suspect this may make for a noticeable difference in performance in use cases involving a large number of argument.

One also needs to worry about the cost of contructing types, though I guess this could be minimal if it is a set. Or should it be the keys of a dict, with the value something meaningful that has to be calculated anyway (like a list of sequence numbers); this may all depend a bit on the implementation of "dance" - the information it gathers might as well get passed on.

For example, suppose np.concatenate() is called on a list of 10,000 dask arrays. Now dask.array.Array.array_function needs to check all arguments to decide whether it can use dask.array.concatenate() or needs to return NotImplemented. By using the types argument, it only needs to do isinstance() checks on the single argument in types, rather than all 10,000 overloaded function arguments

It is probably a good idea to add some of these considerations to the NEP.

-- Marten -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180603/a902d5b4/attachment.html>

More information about the NumPy-Discussion mailing list