[Python-Dev] Python-versus-CPython question for mul dispatch (original) (raw)

Nathaniel Smith njs at pobox.com
Fri May 15 08:53:35 CEST 2015

On Thu, May 14, 2015 at 9:29 PM, Guido van Rossum <guido at python.org> wrote:

I expect you can make something that behaves like list by defining mul and rmul and returning NotImplemented.

Hmm, it's fairly tricky, and part of the trick is that you can never return NotImplemented (because you have to pretty much take over and entirely replace the normal dispatch rules inside mul and rmul), but see attached for something I think should work.

So I guess this is just how Python's list, tuple, etc. work, and PyPy and friends need to match...

-n

On Thursday, May 14, 2015, Stefan Richthofer <Stefan.Richthofer at gmx.de> wrote:

>>Should I be worried? You mean should I be worried ;) Stuff like this is highly relevant for JyNI, so thanks very much for clarifying this subtle behavior. It went onto my todo-list right now to ensure that JyNI will emulate this behavior as soon as I am done with gc-support. (Hopefully it will be feasible, but I can only tell in half a year or so since there are currently other priorities.) Still, this "essay" potentially will save me a lot of time. So, everybody please feel encouraged to post things like this as they come up. Maybe there could be kind of a pitfalls-page somewhere in the docs collecting these things. Best Stefan

> Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr > Von: "Nathaniel Smith" <njs at pobox.com> > An: "Python Dev" <python-dev at python.org> > Betreff: [Python-Dev] Python-versus-CPython question for mul > dispatch > > Hi all, > > While attempting to clean up some of the more squamous aspects of > numpy's operator dispatch code [1][2], I've encountered a situation > where the semantics we want and are using are possible according to > CPython-the-interpreter, but AFAICT ought not to be possible according > to Python-the-language, i.e., it's not clear to me whether it's > possible even in principle to implement an object that works the way > numpy.ndarray does in any other interpreter. Which makes me a bit > nervous, so I wanted to check if there was any ruling on this. > > Specifically, the quirk we are relying on is this: in CPython, if you do > > [1, 2] * myobject > > then myobject's rmul gets called before list.mul, > regardless of the inheritance relationship between list and > type(myobject). This occurs as a side-effect of the weirdness > involved in having both tpasnumber->nbmultiply and > tpassequence->sqrepeat in the C API -- when evaluating "a * b", > CPython tries a's nbmultiply, then b's nbmultiply, then a's > sqrepeat, then b's sqrepeat. Since list has an sqrepeat but not an > nbmultiply, this means that myobject's nbmultiply gets called > before any list method. > > Here's an example demonstrating how weird this is. list.mul wants > an integer, and by "integer" it means "any object with an index > method". So here's a class that list is happy to be multiplied by -- > according to the ordinary rules for operator dispatch, in the example > below Indexable.mul and rmul shouldn't even get a look-in: > > In [3]: class Indexable(object): > ...: def index(self): > ...: return 2 > ...: > > In [4]: [1, 2] * Indexable() > Out[4]: [1, 2, 1, 2] > > But, if I add an rmul method, then this actually wins: > > In [6]: class IndexableWithMul(object): > ...: def index(self): > ...: return 2 > ...: def mul(self, other): > ...: return "indexable forward mul" > ...: def rmul(self, other): > ...: return "indexable reverse mul" > > In [7]: [1, 2] * IndexableWithMul() > Out[7]: 'indexable reverse mul' > > In [8]: IndexableWithMul() * [1, 2] > Out[8]: 'indexable forward mul' > > NumPy arrays, of course, correctly define both index method (which > raises an array on general arrays but coerces to int for arrays that > contain exactly 1 integer), and also defines an nbmultiply slot which > accepts lists and performs elementwise multiplication: > > In [9]: [1, 2] * np.array(2) > Out[9]: array([2, 4]) > > And that's all great! Just what we want. But the only reason this is > possible, AFAICT, is that CPython 'list' is a weird type with > undocumented behaviour that you can't actually define using pure > Python code. > > Should I be worried? > > -n > > [1] https://github.com/numpy/numpy/pull/5864 > [2] https://github.com/numpy/numpy/issues/5844 > > -- > Nathaniel J. Smith -- http://vorpus.org _> ________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de >

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (on iPad)

-- Nathaniel J. Smith -- http://vorpus.org -------------- next part -------------- A non-text attachment was scrubbed... Name: sequence.py Type: text/x-python Size: 2868 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-dev/attachments/20150514/56a62e7e/attachment.py>

More information about the Python-Dev mailing list