[Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support) (original) (raw)

PJ Eby pje at telecommunity.com
Sat May 25 16:08:20 CEST 2013

On Sat, May 25, 2013 at 8:08 AM, Łukasz Langa <lukasz at langa.pl> wrote:

The most important change in this version is that I introduced ABC support and completed a reference implementation.

Excellent! A couple of thoughts on the implementation...

While the dispatch() method allows you to look up what implementation would be selected for a target type, it does not let you figure out whether a particular method has been registered for a type.

That is, if I have a class MyInt that subclasses int, I can't use dispatch() to check whether a MyInt implementation has been registered, because I might get back an implementation registered for int or object. ISTM there should be some way to get at the raw registration info, perhaps by exposing a dictproxy for the registry.

Second, it should be possible to memoize dispatch() using a weak key dictionary that is cleared if new ABC implementations have been registered or when a call to register() is made. The way to detect ABC registrations is via the ABCMeta._abc_invalidation_counter attribute: if its value is different than the previous value saved with the cache, the cache must be cleared, and the new value stored.

(Unfortunately, this is a private attribute at the moment; it might be a good idea to make it public, however, because it's needed for any sort of type dispatching mechanism, not just this one particular generic function implementation.)

Anyway, doing the memoizing in the wrapper function should bring the overall performance very close to a hand-written type dispatch. Code might look something like:

# imported inside closure so that functools module
# doesn't force import of these other modules:
#
from weakref import ref, WeakKeyDictionary
from abc import ABCMeta

cache = WeakKeyDictionary()
valid_as_of = ABCMeta._abc_invalidation_counter

def wrapper(*args, **kw):
    nonlocal valid_as_of
    if valid_as_of != ABCMeta._abc_invalidation_counter:
        cache.clear()
        valid_as_of = ABCMeta._abc_invalidation_counter
    cls = args[0].__class__
    try:
        impl = cache.data[ref(cls)]
    except KeyError:
        impl = cache[cls] = dispatch(cls)
    return impl(*args, **kw)

def register(typ, func=None):
    ...
    cache.clear()
    ...

This would basically eliminate doing any extra (Python) function calls in the common case, and might actually be faster than my current simplegeneric implementation on PyPI (which doesn't even do ABCs at the moment).

More information about the Python-Dev mailing list