[Python-Dev] Python-versus-CPython question for mul dispatch (original) (raw)

Stefan Richthofer Stefan.Richthofer at gmx.de
Fri May 15 04:43:05 CEST 2015

Should I be worried?

You mean should I be worried ;)

Stuff like this is highly relevant for JyNI, so thanks very much for clarifying this subtle behavior. It went onto my todo-list right now to ensure that JyNI will emulate this behavior as soon as I am done with gc-support. (Hopefully it will be feasible, but I can only tell in half a year or so since there are currently other priorities.) Still, this "essay" potentially will save me a lot of time.

So, everybody please feel encouraged to post things like this as they come up. Maybe there could be kind of a pitfalls-page somewhere in the docs collecting these things.

Best

Stefan

Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr Von: "Nathaniel Smith" <njs at pobox.com> An: "Python Dev" <python-dev at python.org> Betreff: [Python-Dev] Python-versus-CPython question for mul dispatch

Hi all, While attempting to clean up some of the more squamous aspects of numpy's operator dispatch code [1][2], I've encountered a situation where the semantics we want and are using are possible according to CPython-the-interpreter, but AFAICT ought not to be possible according to Python-the-language, i.e., it's not clear to me whether it's possible even in principle to implement an object that works the way numpy.ndarray does in any other interpreter. Which makes me a bit nervous, so I wanted to check if there was any ruling on this. Specifically, the quirk we are relying on is this: in CPython, if you do [1, 2] * myobject then myobject's rmul gets called before list.mul, regardless of the inheritance relationship between list and type(myobject). This occurs as a side-effect of the weirdness involved in having both tpasnumber->nbmultiply and tpassequence->sqrepeat in the C API -- when evaluating "a * b", CPython tries a's nbmultiply, then b's nbmultiply, then a's sqrepeat, then b's sqrepeat. Since list has an sqrepeat but not an nbmultiply, this means that myobject's nbmultiply gets called before any list method. Here's an example demonstrating how weird this is. list.mul wants an integer, and by "integer" it means "any object with an index method". So here's a class that list is happy to be multiplied by -- according to the ordinary rules for operator dispatch, in the example below Indexable.mul and rmul shouldn't even get a look-in: In [3]: class Indexable(object): ...: def index(self): ...: return 2 ...: In [4]: [1, 2] * Indexable() Out[4]: [1, 2, 1, 2] But, if I add an rmul method, then this actually wins: In [6]: class IndexableWithMul(object): ...: def index(self): ...: return 2 ...: def mul(self, other): ...: return "indexable forward mul" ...: def rmul(self, other): ...: return "indexable reverse mul" In [7]: [1, 2] * IndexableWithMul() Out[7]: 'indexable reverse mul' In [8]: IndexableWithMul() * [1, 2] Out[8]: 'indexable forward mul' NumPy arrays, of course, correctly define both index method (which raises an array on general arrays but coerces to int for arrays that contain exactly 1 integer), and also defines an nbmultiply slot which accepts lists and performs elementwise multiplication: In [9]: [1, 2] * np.array(2) Out[9]: array([2, 4]) And that's all great! Just what we want. But the only reason this is possible, AFAICT, is that CPython 'list' is a weird type with undocumented behaviour that you can't actually define using pure Python code. Should I be worried? -n [1] https://github.com/numpy/numpy/pull/5864 [2] https://github.com/numpy/numpy/issues/5844 -- Nathaniel J. Smith -- http://vorpus.org

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de

More information about the Python-Dev mailing list