[Python-Dev] PEP 447: hooking into super() attribute resolution (original) (raw)

Ronald Oussoren ronaldoussoren at mac.com
Mon Jul 15 17:12:45 CEST 2013

Hi,

I've posted a new update of my proposal to add a way to override the attribute resolution proces by super(). I've rewritten the PEP and implementation based on feedback by Steve Dower.

In the current edition of the proposal the hook is a method on the type, defined in a metaclass for types. A simple demo:

class meta (type):
    def __locallookup__(cls, name):
        return type.__locallookup__(cls, name.upper())

class S (metaclass=meta):
    def m(self):
        return 42

    def M(self):
         return "fortytwo"

class SS (S):
    def m(self):
        return 21

    def M(self):
       return "twentyone"

o = SS()
print("self", o.m())
print("super", super(SS, o).m())

The last line currently prints "super 42" and would print "super fortytwo" with my proposal.

A major open issue: the locallookup method could also be used for normal attribute resolution, but that probably causes issues with attribute caching (see the PEP for details). I haven't worked out yet if it is worthwhile to tweak the proposal to fix the caching issues (even though the proposal currently says that PyObject_GenericGetAttr will use the new method). Fixing the caching issue definitly would help in my primary use case by reducing code duplication, but might end up making the API unnecessarily complex.

Ronald

PEP: 447 Title: Hooking into super attribute resolution Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Ronald Oussoren <ronaldoussoren at mac.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Jun-2013 Post-History: 2-Jul-2013, ?

Abstract

In current python releases the attribute resolution of the super class_ peeks in the __dict__ attribute of classes on the MRO to look for attributes. This PEP introduces a hook that classes can use to override that behavior for specific classes.

Rationale

Peeking in the class __dict__ works for regular classes, but can cause problems when a class dynamicly looks up attributes in a __getattribute__ method.

This new hook makes it possible to affect attribute lookup for both normal attribute lookup and lookup through the super class_.

The superclass attribute lookup hook

Both super.__getattribute__ and object.__getattribute__ (or PyObject_GenericGetAttr_ in C code) walk an object's MRO and peek in the class' __dict__ to look up attributes. A way to affect this lookup is using a method on the meta class for the type, that by default looks up the name in the class __dict__.

In Python code

A meta type can define a method __locallookup__ that is called during attribute resolution by both super.__getattribute__ and object.__getattribute::

class MetaType (type):

    def __locallookup__(cls, name):
        try:
            return cls.__dict__[name]
        except KeyError:
            raise AttributeError(name) from None

The example method above is pseudo code for the implementation of this method on type_ (the actual implementation is in C, and doesn't suffer from the recursion problem in this example).

The __locallookup__ method has as its arguments a class and the name of the attribute that is looked up. It should return the value of the attribute without invoking descriptors, or raise AttributeError_ when the name cannot be found.

In C code

A new slot tp_locallookup is added to the PyTypeObject struct, this slot corresponds to the __locallookup__ method on type_.

The slot has the following prototype::

PyObject* (*locallookupfunc)(PyTypeObject* cls, PyObject* name);

This method should lookup name in the namespace of cls, without looking at superclasses, and should not invoke descriptors. The method returns NULL without setting an exception when the name cannot be found, and returns a new reference otherwise (not a borrowed reference).

Usage of this hook

The new method will be defined for type_, and will peek in the class dictionary::

static PyObject*
type_locallookup(PyTypeObject* cls, PyObject* name)
{
    PyObject* res;
    if (!cls->tp_dict) {
    return NULL;
}

res = PyDict_GetItem(cls, name);
Py_XINCREF(res);
return res;
}

The new method will be used by both PyObject_GenericGetAttr_ and super.__getattribute__ instead of peeking in a type's tp_dict.

Alternative proposals

__getattribute_super__ ..........................

An earlier version of this PEP used the following static method on classes::

def __getattribute_super__(cls, name, object, owner): pass

This method performed name lookup as well as invoking descriptors and was necessarily limited to working only with super.__getattribute__.

Reuse tp_getattro .....................

It would be nice to avoid adding a new slot, thus keeping the API simpler and easier to understand. A comment on Issue 18181_ asked about reusing the tp_getattro slot, that is super could call the tp_getattro slot of all methods along the MRO.

That won't work because tp_getattro will look in the instance __dict__ before it tries to resolve attributes using classes in the MRO. This would mean that using tp_getattro instead of peeking the class dictionaries changes the semantics of the super class_.

Open Issues

References

Copyright

This document has been placed in the public domain.

.. _Issue 18181: http://bugs.python.org/issue18181

.. _super class: http://docs.python.org/3/library/functions.html#super

.. _NotImplemented: http://docs.python.org/3/library/constants.html#NotImplemented

.. _PyObject_GenericGetAttr: http://docs.python.org/3/c-api/object.html#PyObject_GenericGetAttr

.. _type: http://docs.python.org/3/library/functions.html#type

.. _AttributeError: http://docs.python.org/3/library/exceptions.html#AttributeError

More information about the Python-Dev mailing list