[Numpy-discussion] Proposal to accept NEP-18, array_function protocol (original) (raw)

einstein.edison at gmail.com einstein.edison at gmail.com
Tue Aug 21 04:53:54 EDT 2018

I’m +0 on removing it, so mostly neutral, but slightly in favour. While I see the argument for having it, I also see it as a violation of DRY... The information is already available in relevant arguments.

I doubt any people implementing this protocol are going to be lazy enough not to implement a type check. So far, we’ve been good on array_ufunc.

The part of me that says it’s good to have it is the mostly “squeeze every bit of performance out” part.

Best Regards Hameer Abbasi Sent from my iPhone

On 21. Aug 2018, at 10:34, Marten van Kerkwijk <m.h.vankerkwijk at gmail.com> wrote:

>> I don't really understand the 'types' frozenset. The NEP says "it will >> be used by most arrayfunction methods, which otherwise would need >> to extract this information themselves"... but they still need to >> extract the information themselves, because they still have to examine >> each object and figure out what type it is. And, simply creating a >> frozenset costs ~0.2 µs on my laptop, which is overhead that we can't >> possibly optimize later... > > > The most flexible alternative would be to just say that we provide an > fixed-length iterable, and return a tuple object. (In my microbenchmarks, > it's faster to make a tuple than a list or set.) In an early draft of the > NEP, I proposed exactly this, but the speed difference seemed really > marginal to me. > > I included 'types' in the interface because I really do think it's something > that almost all arrayfunction implementations should use use. It > preserves a nice separation of concerns between dispatching logic and > implementations for a new type. At least as long as arrayfunction is > experimental, I don't think we should be encouraging people to write > functions that could return NotImplemented directly and to rely entirely on > the NumPy interface. > > Many but not all implementations will need to look at argument types. This > is only really essential for cases where mixed operations between NumPy > arrays and another type are allowed. If you only implement the NumPy > interface for MyArray objects, then in the usual Python style you wouldn't > need isinstance checks. > > It's also important from an ecosystem perspective. If we don't make it easy > to get type information, my guess is that many arrayfunction authors > wouldn't bother to return NotImplemented for unexpected types, which means > that arrayfunction will break in weird ways when used with objects from > unrelated libraries. This is much more of a detail as compared to the rest of the discussion, so I don't want to quibble too much about it. (Especially since if we keep things really-provisional, we can change our mind about the argument later :-).) Mostly I'm just confused, because there are lots of dunder functions in Python (and NumPy), and none of them take a special 'types' argument... so what's special about arrayfunction that makes it necessary/worthwhile? Any implementation of, say, concatenate-via-arrayfunction is going to involve iterating through all the arguments and looking at each of them to figure out what kind of object it is and how to handle it, right? That's true whether or not they've done a "pre-check" using the types set, so in theory it's just as easy to return NotImplemented at that point. But I guess your point in the last paragraph is that this means there will be lots of chances to mess up the NotImplemented-returning code in particular, especially since it's less likely to be tested than the happy path, which seems plausible. So basically the point of the types set is to let people factor out that little bit of lots of functions into one common place? I guess some careful devs might be unhappy with paying extra so that other lazier devs can get away with being lazy, but maybe it's a good tradeoff for us (esp. since as numpy devs, we'll be getting the bug reports regardless :-)). If that's the goal, then it does make me wonder if there might be a more direct way to accomplish it -- like, should we let classes define an arrayfunctiontypes attribute that numpy would check before even trying to dispatch to arrayfunction? I quite like that idea; I've not been enchanted by the extra types either - it seems like method in _arrayufunc_, it could become quite superfluous. -- Marten

NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180821/d32fbf4a/attachment.html>

More information about the NumPy-Discussion mailing list