[Python-Dev] NotImplemented reaching top-level (original) (raw)

M.-A. Lemburg mal at egenix.com
Thu Dec 29 15:40:36 CET 2005

Hi Armin,

On Wed, Dec 28, 2005 at 09:56:43PM +0100, M.-A. Lemburg wrote:

d += 1.2 d NotImplemented The PEP documenting the coercion logic has complete tables for what should happen: Well, '+=' does not invoke coercion at all, with new-style classes like Decimal.

True, it doesn't invoke coercion in the sense that a coercion method is called, but the mechanism described in the PEP is still used via PyNumber_InPlaceAdd().

Looking at the code in abstract.c the above problem appears to be related to the special cases applied to += and *= in case both operands cannot deal with the type combination.

In such a case, a check is done whether the operation could be interpreted as sequence operation (concat or repeat) and then delegated to the appropriate handlers. Indeed. The bug was caused by this delegation, which (prior to my patch) would also return a PyNotImplemented that would leak through abstract.c. My patch is to remove this unnecessary delegation by not defining sqconcat/sqrepeat for user-defined classes, and restoring the original expectation that the sqconcat/sqrepeat slots should not return PyNotImplemented. How does this relate to coercion?

The Py_NotImplemented singleton was introduced in the coercion proposal to mean "there is no implementation to execute the requested operation on the given combination of types".

At the time we also considered using an exception for this, but it turned out that this caused too much of a slow-down. Hence the use of a special singleton which could be tested for by a simple pointer comparison.

Originally, the singleton was only needed for mixed-type operations. It seems that its use has spread to other areas as well and can now also refer to missing same-type operator implementations.

But then again, looking in typeobject.c, the following code could be the cause for leaking a NotImplemented singleton reference:

_#define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, OPSTR, ROPSTR) _ _static PyObject * _ _FUNCNAME(PyObject *self, PyObject *other) _ _{ _ _static PyObject *cachestr, *rcachestr; _ _int doother = self->obtype != other->obtype && _ _other->obtype->tpasnumber != NULL && _ _other->obtype->tpasnumber->SLOTNAME == TESTFUNC; _ _if (self->obtype->tpasnumber != NULL && _ _self->obtype->tpasnumber->SLOTNAME == TESTFUNC) { _ _PyObject *r; _ _if (doother && _ _PyTypeIsSubtype(other->obtype, self->obtype) && _ _methodisoverloaded(self, other, ROPSTR)) { _ _r = callmaybe( _ _other, ROPSTR, &rcachestr, "(O)", self); _ _if (r != PyNotImplemented) _ _return r; _ _PyDECREF(r); _ _doother = 0; _ _} _ _r = callmaybe( _ _self, OPSTR, &cachestr, "(O)", other); _ _if (r != PyNotImplemented || _ _other->obtype == self->obtype) _ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If both types are of the same type, then a NotImplemented returng value would be returned. Indeed, however: _return r; _ _PyDECREF(r); _ _} _ _if (doother) { _ _return callmaybe( _ _other, ROPSTR, &rcachestr, "(O)", self); _ _} _ _PyINCREF(PyNotImplemented); _ _return PyNotImplemented; _ } This last statement also returns PyNotImplemented. So it's expected of this function to be able to return PyNotImplemented, isn't it? The type slots like nbadd can return PyNotImplemented; the code that converts it to a TypeError is in the caller, which is abstract.c.

You're right - silly me.

Regards,

Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Dec 29 2005)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Python-Dev mailing list