[Python-Dev] Possible rough edges in Python 3 metaclasses (was Re: Language reference updated for metaclasses) (original) (raw)

PJ Eby pje at telecommunity.com
Tue Jun 5 02:10:33 CEST 2012

On Mon, Jun 4, 2012 at 7:18 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

On Tue, Jun 5, 2012 at 8:58 AM, PJ Eby <pje at telecommunity.com> wrote: > On Mon, Jun 4, 2012 at 6:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: >> >> It's actually the pre-decoration class, since the cell is initialised >> before the class is passed to the first decorator. I agree it's a little >> weird, but I did try to describe it accurately in the new docs. > > I see that now; it might be helpful to explicitly call that out. > > This is adding to my list of Python 3 metaclass gripes, though. In Python > 2, I have in-the-body-of-a-class decorators implemented using metaclasses, > that will no longer work because of PEP 3115...

I'm not quite following this one - do you mean they won't support prepare, won't play nicely with other metaclasses that implement prepare, or something else?

I mean that class-level metaclass is no longer supported as of PEP 3115, so I can't use that as a way to non-invasively obtain the enclosing class at class creation time.

(Unfortunately, I didn't realize until relatively recently that it wasn't supported any more; the PEP itself doesn't say the functionality will be removed. Otherwise, I'd have lobbied sooner for a better migration path.)

> Meanwhile, mixing metaclasses is more difficult than ever, due to > prepare, and none of these flaws can be worked around officially, > because buildclass is an "implementation detail". I really want to > like Python 3, but am still hoping against hope for the restoration of hooks > that metaclass allowed, or some alternative mechanism that would serve > the same use cases. > > Specifically, my main use case is method-level decorators and attribute > descriptors that need to interact with a user-defined class, without > requiring that user-defined class to either 1) redundantly decorate the > class or 2) inherit from some specific base or inject a specific metaclass. > I only use metaclass in 2.x for this because it's the only way for code > executed in a class body to gain access to the class at creation time. > > The reason for wanting this to be transparent is that 1) if you forget the > redundant class-decorator, mixin, or metaclass, stuff will silently not > work,

Why would it silently not work? What's preventing you from having decorators that create wrapped functions that fail noisily when called, then providing a class decorator that unwraps those functions, fixes them up with the class references they need and stores the unwrapped and updated versions back on the class.

If you are registering functions or attributes in some type of registry either at the class level or in some sort of global registry, then the function will never be called or the attribute accessed, so there is no opportunity to give an error message saying, "you should have done this". Something will simply fail to happen that should have happened.

Essentially, the lack of this hook forces things to be done outside of class bodies that make more sense to be done inside class bodies.

and 2) mixing bases or metaclasses has much higher coupling to the > library providing the decorators or descriptors, and greatly increases the > likelihood of mixing metaclasses.

So don't do that, then. Be explicit.

Easy for you to say. However, I have many decorators and descriptors which follow this pattern, in various separately-distributed libraries. If each of these libraries now grows a redundant class decorator for Python 3, then any code which uses more than one in the same class will now have a big stack of decorator noise on top of it... and missing any one of them will cause silent failure.

And of course, all other libraries which use these decorators and descriptors will also have to add this line-noise in order to port to Python 3... including the applications that depend on those libraries. And let's not forget the people who subclass or extend any of my decorators or descriptors -- they will need to tell their users to begin adding redundant class-level decorators, and so on.

> And at the moment, the only workaround I can come up with that doesn't > involve replacing buildclass is abusing the system trace hook; ISTM > that replacing buildclass is the better of those two options.

Stop trying to be implicit. Implicit magic sucks,

The fact that a method-level decorator or descriptor might need access to its containing class when the class is created does not in any way make it intrinsically "magical" or "implicit". The focus here is on the decorator or descriptor: the class access is just an implementation detail, and generally doesn't involve modifying the class itself.

So, please let's not start FUDding here. None of the descriptors or decorators I'm describing are implicit or magical in the least. They are documented as doing what they do, and they use perfectly legal mechanisms of Python 2 to implement those behaviors. If tomorrow a new PEP were introduced that provided an alternate way of doing this, I'd gladly use it to migrate these features forward.

Implying that my use cases "suck" does not help; I could just as easily say, "redundancy sucks", and we are no further along to a solution. It might be more helpful to propose some alternate mechanism to provide the same feature to be introduced in a future Python version, like a special decorators key in a class dictionary that the default type would iterate over and call with the class. Or perhaps a proposal that the default type creation code simply iterate over the contents of the class namespace and call "ob.used_in_class(cls, attrname)" on each found object. Or something else altogether, any of which I would happily use in place of the old metaclass hook, and which would certainly be less obscure.

At this point, with the additions of types.newclass(), ISTM that every > Python implementation will have to have a buildclass function or its > equivalent; all that remains is the question of whether they allow > replacing it.

types.newclass() is actually a pure Python reimplementation of the PEP 3115 algorithm. Why would it imply the need for a buildclass function?

Because now every Python implementation will contain the code to do this twice: once inside its normal class creation machinery, and once inside of types.new_class(). (Well, unless they share code internally, of course.)

IOW, saying that build_class is an implementation detail of CPython doesn't quite wash: thanks to types.new_class() every Python implementation now must support programmatically creating a class in this way. The only detail that remains is whether it's allowed to replace the built-in equivalent to build_class, and whether it's documented as a standard feature ala import (and its metaclass predecessor).

Note, though, that I've only focused on build_class because it's low-hanging fruit: it could be used to work around the absence of metaclass or a more-specific hook like the ones I mentioned above, and it's already implemented in Python 3.x. (That doesn't make it the best solution, of course, just a low-hanging one.) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20120604/ac4931de/attachment.html>

More information about the Python-Dev mailing list