[Python-Dev] PEP 487 vs 422 (dynamic class decoration) (original) (raw)

PJ Eby pje at telecommunity.com
Fri Apr 3 03:32:29 CEST 2015

On Thu, Apr 2, 2015 at 6:24 PM, Martin Teichmann <lkb.teichmann at gmail.com> wrote:

The whole point of PEP 487 was to reduce PEP 422 so much that it can be written in python and back-ported.

As I said earlier, it's a fine feature and should be in the stdlib for Python 3. (But it should have a noconflict feature added, and it doesn't need a language change.)

However, since my specific use case was the one PEP 422 was originally written to solve, and PEP 487 does not address that use case, it is not a suitable substitute for PEP 422.

This is also not your fault; you didn't force Nick to withdraw it, after all. ;-)

My main concern in this thread, however, is ensuring that either the use case behind PEP 422 doesn't get dropped, or that Nick is now okay with me implementing that feature by monkeypatching build_class. Since he practically begged me not to do that in 2012, and IIRC specifically created PEP 422 to provide an alternative way for me to accomplish this specific use case, I wanted to see what his current take was. (That is, did he forget the history of the PEP, or does he no longer care about userspace code hooking build_class? Is there some other proposal that would be a viable alternative? etc.)

Now you want to be able to write decorators whose details are filled in at class creation time.

Not "now"; it's been possible to do this in Python 2 for over a decade, and code that does so is in current use by other packages. The package providing this feature (DecoratorTools) was downloaded 145 times today, and 3274 times in the past month, so there is active, current use of it by other Python 2 packages. (Though I don't know how many of them depend directly or indirectly upon this particular feature.)

Currently, however, it is not possible to port this feature of DecoratorTools (or any other package that uses that feature, recursively) to Python 3, due to the removal of metaclass and the lack of any suitable substitute hook.

Your point is that you want to be able to use your decorators without having to ask users to also inherit a specific class. I personally don't think that's desirable. Many frameworks out there have such kind of decorators and mandatory base classes and that works fine.

The intended use case is for generic method decorators that have nothing to do with the base class per se, so inheriting from a specific base-class is an anti-feature in this case.

The only problem remains once you need to inherit more than one of those classes, as their metaclasses most likely clash. This is what PEP 487 fixes.

No, it addresses the issue for certain specific metaclass use cases. It does not solve the problem of metaclass conflict in general; for that you need something like the sample noconflict code I posted, which works for Python 3.1+ and doesn't require a language change.

So my opinion is that it is not too hard a requirement to ask a user to inherit a specific mixin class for the sake of using a decorator.

If this logic were applied to PEP 487 as it currently stands, the PEP should be rejected, since its use case is even more easily accomplished by inheriting from a specific mixin class. (Since the feature only works on subclasses anyway!)

Further, if the claim is that metaclass conflict potential makes PEP 487 worthy of a language change, then by the same logic method decorators are just as worthy of a language change, since any mixin required to use a method decorator would be just as susceptible to metaclass conflicts as SubclassInit. (Notably, the stdlib's ABCMeta is a common cause of metaclass conflicts in Python 2.6+ -- if you mix in anything that implements an ABC by subclassing it, you will get a metaclass conflict.)

Finally, I of course disagree with the conclusion that it's okay to require mixins in order for method decorators to access the containing class, since it is not a requirement in Python 2, due to the availability of the metaclass hook. Further, PEP 422 was previously approved to fix this problem, and has a patch in progress, so I'm understandably upset by its sudden withdrawal and lack of suitable replacement.

So personally, I think that PEP 422 should be un-withdrawn (or replaced with something else), and PEP 487 should be retargeted towards defining a metaclass module for the stdlib, including a noconflict implementation to address metaclass conflict issues. (Mine or someone else's, as long as it works.) PEP 487 should not be a proposal to change the language, as the provided features don't require it. (And it definitely shouldn't pre-empt a separately useful feature that does require a language change.)

At this point, though, I mostly just want to get some kind of closure. After three years, I'd like to know if this is a yea or nay, so I can port the thing and move on, whether it's through a standardized mechanism or ugly monkeypatching. Honestly, the only reason I'm even discussing this in the first place is because 1) Nick pleaded with me three years ago not to hold off porting until a standardized way of doing this could be added to the language, and 2) I've had some recent inquiries from users about porting PEAK-Rules (which uses this particular feature of DecoratorTools) to Python 3. So I went to check on PEP 422's status, and here we are.

More information about the Python-Dev mailing list