[Python-Dev] Possible rough edges in Python 3 metaclasses (was Re: Language reference updated for metaclasses) (original) (raw)

PJ Eby pje at telecommunity.com
Tue Jun 5 17:28:21 CEST 2012

On Tue, Jun 5, 2012 at 3:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

Please don't try to coerce everyone else into supporting such an ugly hack by abusing an implementation detail.

Whoa, whoa there. Again with the FUD.

Sorry if I gave the impression that I'm about to unleash the monkeypatching hordes tomorrow or something. I'm unlikely to begin serious Python 3 porting of the relevant libraries before 3.3 is released; the reason I'm talking about this now is because there's currently Python-Dev discussion regarding metaclasses, so it seemed like a good time to bring the subject up, to see if there were any good solutions.

Just because my three existing options are, well, the only options if I started porting today, doesn't mean I think they're the only options. As I said, an option 4 or 5 would be fantastic, and your new PEP 422 is wonderful in that regard. Thank you VERY much for putting that together, I appreciate that very much.

(I just wish you hadn't felt forced or coerced to do it -- that was not my intention at all!)

Frankly, a big part of my leaning towards the build_class option was that it's the least work by Python implementors (at least in terms of coding) to get my use case met. The reason I didn't write a PEP myself is because I didn't want to load up Python with yet another protocol, when my use case could be met by stuff that's already implemented in CPython.

IOW, my motivation for saying, "hey, can't I just use this nice hook here" was to avoid asking for a new feature, if there weren't enough other people interested in a decoration protocol of this sort.

That is, I was trying to NOT make anybody do a bunch of work on my behalf. (Clearly, I wasn't successful in the attempt, but I should at least get credit for trying. ;-)

Now, one minor annoyance with current class decorators is that they're not inherited. This is sometimes what you want, but sometimes you would prefer to automatically decorate all subclasses as well. Currently, that means writing a custom metaclass to automatically apply the decorators. This has all the problems you have noted with composability.

It seems then, that a potentially clean solution may be found by adding a dynamic class decoration hook. As a quick sketch of such a scheme, add the following step to the class creation process (between the current class creation process, but before the execution of lexical decorators): for mrocls in cls.mro(): decorators = mrocls.dict.get("decorators", ()) for deco in reversed(decorators): cls = deco(cls) Would such a dynamic class decoration hook meet your use case? Such a hook has use cases (specifically involving decorator inheritance) that don't require the use of sys.getframes(), so is far more likely to achieve the necessary level of consensus.

Absolutely. The main challenge with it is that I would need stateful decorators, so that they do nothing when called more than once; the motivating use cases for me do not require actual decorator inheritance. But I do have other stuff I'm currently using metaclasses for in 2.x, that could probably be eliminated with PEP 422. (For example, the #1 metaclass I use in 2.x is one that basically lets classes have class_new and class_init methods: a rough equivalent to in-line metaclasses or inheritable decorators!)

In general, implementing what's effectively inherited decoration is what most metaclasses actually get used for, so PEP 422 is a big step forward in that.

Sketching something to get a feel for the PEP...

def inheritable(*decos): """Wrap a class with inheritable decorators""" def decorate(cls): cls.decorators = list(decos)+list(cls.dict.get('decorators',())) for deco in reversed(decos): cls = deco(cls) return cls

Hm. There are a few interesting consequences of the PEP as written. In-body decorators affect the class closure (and thus super()), but out-of-body decorators don't. By me this is a good thing, but it is a bit of complexity that needs mentioning. Likewise, the need for inheritable decorators to be idempotent, in case two base classes list the same decorator. (For my own use attribute/method use cases, I can just have them remove themselves from the class's decorators upon execution.)

You complain that metaclasses are hard to compose, and your

"solution" is to monkeypatch a deliberately undocumented builtin?

To be clear, what I specifically proposed (as I mentioned in an earlier thread) was simply to patch build_class in order to restore the missing metaclass hook. (Which, incidentally, would make ALL code using metaclass cross-version compatible between 2.x and 3.x: a potentially valuable thing in and of itself!)

As for metaclasses being hard to compose, PEP 422 is definitely a step in the right direction. (Automatic metaclass combining is about the only thing that would improve it any further.) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20120605/e5b2608a/attachment.html>

More information about the Python-Dev mailing list