[Python-Dev] Python-versus-CPython question for mul dispatch (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sat May 16 10:15:56 CEST 2015

On 15 May 2015 at 10:45, Nathaniel Smith <njs at pobox.com> wrote:

Hi all,

While attempting to clean up some of the more squamous aspects of numpy's operator dispatch code [1][2], I've encountered a situation where the semantics we want and are using are possible according to CPython-the-interpreter, but AFAICT ought not to be possible according to Python-the-language, i.e., it's not clear to me whether it's possible even in principle to implement an object that works the way numpy.ndarray does in any other interpreter. Which makes me a bit nervous, so I wanted to check if there was any ruling on this.

It's a known CPython operand precedence bug due to the fact several of the builtin types only implement sq_concat & sq_repeat without implementing nb_add & nb_mul: http://bugs.python.org/issue11477

There's then a related problem where we don't process "NotImplemented" results from sq_concat and sq_repeat properly, so all the builtin sequence types throw TypeError directly, instead of returning NotImplemented when they don't recognise the other type.

I wrote a preliminary patch attempting to fix it a few years back after the issue was discovered by Mike Bayer and Alex Gaynor when porting SQL Alchemy to PyPy, but never committed it because my own verdict on the approach I used was that it rendered the abstract object API implementation for mul and add utterly unmaintainable.

The better fix would be to make defining sq_concat and sq_repeat more like defining add and mul at the Python level: PyType_Ready should implicitly fill in nb_add and nb_mul references to standard implementations that delegate to sq_concat and sq_repeat, and we should update the implementations of the latter for the standard library sequence types implemented in C to return NotImplemented rather than throwing TypeError directly.

However, my intermittent attempts to get anyone else interested in fixing it haven't borne any fruit, and I've prioritised other projects over coming up with a different patch myself.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list