bpo-31333: Re-implement ABCMeta in C by ilevkivskyi · Pull Request #5273 · python/cpython (original) (raw)

@1st1

What exact magic does six use?

six uses this code:

def with_metaclass(meta, *bases): # ... class metaclass(type):

    def __new__(cls, name, this_bases, d):
        return meta(name, bases, d)

    @classmethod
    def __prepare__(cls, name, this_bases):
        return meta.__prepare__(name, bases)

return type.new(metaclass, 'temporary_class', (), {})

The problem is that it doesn't work with C-level metaclasses.

BTW, have you considered implementing caches and instance/subclass hooks in C, but implementing the actual ABCMeta class in pure Python? That way ABCMeta methods would have nice C-accelerated versions, but the 'six' problem should go away.

This actually sounds like a reasonable solution.

I'd go with (1). There're 0 reasons to use those private caches/registries directly.
I had a similar problem problem in asyncio in 3.6, when I debated whether I want to expose private Task and Future attributes or not. Turns out that we'll be hiding some of them in 3.7 because it's impossible to optimize/refactor code otherwise.

OK, I can hide them (we then just need to update code in refleak.py that explicitly reads them).
How do you propose to have the hidden C-level attributes for Python level class? Just have a single huge dictionary, so that it will work like this:

pseudo-code, will be in C

_the_registry: Dict[WeakRef[type], Set[WeakRef[type]]] = {} ... def _abc_register(cls, subcls): _registry = _the_registry[ref(cls)] _registry.add(ref(subcls)) return subcls

or is there a better way?

Sounds like "minor downside that dead references will stay in caches" is a backwards-incompatible change...

Yes, I have a TODO about limiting cache growth in code, but if we are going to hide private cache attributes, then it is easy, we can just register callbacks since caches are never iterated in this code.

We can also use WeakSet and WeakKeyDictionary btw.

As @methane mentioned they are implemented in Python and also slow and overly general. As I said above, if we are going to hide the attributes, then we don't need this for caches. We only iterate over the registry and for it I can just use callbscks with a "commit queue" and an iteration guard (this is actually the idea behind WeakSet, but we can make it much simpler since we are doing very limited set of operations with the registry).