[Numpy-discussion] Proposal to accept NEP-18, array_function protocol (original) (raw)

Marten van Kerkwijk m.h.vankerkwijk at gmail.com
Wed Aug 29 09:44:23 EDT 2018

HI All,

On the backwards compatibility: from an astropy perspective, I would expect that the introduction of __array_function__ implies a guarantee that the functionality it provides will remain, i.e., that it will continue to be possible to override, say, concatenate. It is not a big deal if the actual implementation changes (say, __array_concatenate__ is introduced) and we need to do some version-dependent magic to ensure that our users do not notice the change; we did the same with __array_ufunc__ - the wrap/prepare machinery will be gone only in the next version of astropy, when our minimum version for numpy reaches 1.13.

One thing perhaps worth remembering that what really matters is not so much __array_function__ but the set of functions that actually implement the override. It would seem unlikely this would be the complete numpy API on the first go; perhaps it is an idea to be somewhat specific about which part we start with? My vote would go towards array manipulation routines like concatenate. I would also suggest to avoid those functions for which ufunc implementations would seem quite possible (i.e., avoid things like median, etc.)

All the best,

Marten

On Wed, Aug 29, 2018 at 8:31 AM Matthew Rocklin <mrocklin at gmail.com> wrote:

>> 1. if we do find ourselves in a situation where changing this would break lots of users, will we consider ourselves beholden to them?

I think that it would be useful for Numpy's continued evolution to develop the ability to include code on a provisional basis. Other projects do this and they just have big bold "Experimental" notes everywhere that a user might go to learn about the functionality (docs, docstrings, blogposts). Some users will definitely get burned, yes, but the alternative is to burn all other users a little bit by moving too slowly and not trying things out. This is different from how Numpy has operated in the past, but that might be ok. >> 2. is it plausible that we'll find ourselves in that situation? > Personally, for as long as this protocol is experimental, I’ll add a warning in the docs of sparse as well; saying this might disappear anytime Yup. Same for Dask. We're pretty accustomed to this. On Wed, Aug 29, 2018 at 7:01 AM Hameer Abbasi <einstein.edison at gmail.com> wrote:

> On 29. Aug 2018, at 11:44, Matti Picus <matti.picus at gmail.com> wrote: > > On 29/08/18 10:37, Nathaniel Smith wrote: >> it's easy to imagine scenarios where the >> people being broken aren't the ones who had a chance to read the docs >> – e.g. if a major package starts relying on arrayfunction, then >> it's alltheir users who we'd be breaking, even though they had >> nothing to do with it. > This is a packaging problem. This proposal is intended for use by other "major packages", not so much for end-users. We would have much more trouble if we were proposing a broad change to something like indexing or the random number module (see those NEPs). If we break one of those major packages, it is on them to pin the version of NumPy they can work with. In my opinion very few end users will be implementing their own ndarray classes with _arrayfunction_. While we will get issue reports, we can handle them much as we do the MKL or OpenBLAS ones - pinpoint the problem and urge users to complain to those packages.

One thing that might help here is nightly or continuous CI builds of the NumPy wheels. This would be good, as we could test it in CI, and fix it when it comes up. But I guess that’s another discussion. Personally, for as long as this protocol is experimental, I’ll add a warning in the docs of sparse as well; saying this might disappear anytime. > > Other than adding a warning, I am not sure what the concrete proposal is here. To not accept the NEP? > Matti _> ________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion

NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion

NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180829/609fc462/attachment-0001.html>

More information about the NumPy-Discussion mailing list