[Python-Dev] PEP 487 vs 422 (dynamic class decoration) (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sun Apr 5 03:33:36 CEST 2015

On 4 April 2015 at 06:36, PJ Eby <pje at telecommunity.com> wrote:

On Fri, Apr 3, 2015 at 4:21 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

No, you can't do it currently without risking a backwards incompatibility through the introduction of a custom metaclass. Right... which is precisely why I'm suggesting the noconflict() metaclass factory function as a general solution for providing useful metaclasses, and why I think that PEP 487 should break the namespacing and subclass init features into separate metaclasses, and add that noconflict feature. It will then become a good example for people moving forward writing metaclasses. Basically, as long as you don't have the pointless conflict errors, you can write co-operative metaclass mixins as easily as you can write regular co-operative mixins. I was missing this point myself because I've been too steeped in Python 2's complexities: writing a usable version of noconflict() is a lot more complex and its invocation far more obscure. In Python 2, there's classic classes, class- and module-level metaclass, ExtensionClass, and all sorts of other headaches for automatic mixing. In Python 3, though, all that stuff goes out the window, and even my 90-line version that's almost half comments is probably still overengineered compared to what's actually needed to do the mixing.

D'oh, I had the same problem you did - I'd been assuming this was entirely infeasible because of all the complexities it involved back in Python 2, and had never adequately reconsidered the question in a PEP 3115 based world :(

So actually reading https://gist.github.com/pjeby/75ca26f8d2a7a0c68e30 properly, you're starting to convince me that a "noconflict" metaclass resolver would be a valuable and viable addition to the Python 3 type system machinery.

The future possible language level enhancement would then be to make that automatic resolution of metaclass conflicts part of the default metaclass determination process. I realise you've been trying to explain that to me for a few days now, I'm just writing it out explicitly to make it clear I finally get it :)

Given my change of heart, I believe that at this point, if you were willing to champion a revived PEP 422 that implemented the behaviour you're after, that would be ideal, with monkeypatching the desired behaviour in as a fallback plan if the PEP is still ultimately rejected. Alternatively, you could go the monkeypatching path first, and then potentially seek standardisation later after you've had some practical experience with it - I now consider it an orthogonal capability to the feature in PEP 487, so the acceptance of the latter wouldn't necessarily preclude acceptance of a hook for class postprocessing injection. A lot of things have changed since the original discussion, mostly in the direction of me having even less time for Python work than previously, so it's unlikely that me championing a PEP is a realistic possibility. Frankly, I'm immensely fatigued at the discussion already, and the need to go over the same questions a third time seems like not something I'm going to want to put energy into.

Heh, one of the main reasons PEP 422 ended up languishing for so long is that I started putting more time into other projects (PyPA, the PSF, the import system, Python 3 advocacy, etc), so with both you & me occupied elsewhere, we didn't really have anyone driving the discussion forward on the metaclass machinery side of things. Martin's very pertinent challenges to some of the unnecessary design complexity in PEP 422 has noticeably changed that dynamic for the better :)

However it sounds like there is some growing consensus towards the idea of simply notifying interested class members of their class membership, so if there ends up being a consensus to standardize that protocol and what part of the class-building process it gets invoked in, then I will implement a backport (or use such a backport if someone else implements it), when I actually start porting my libraries to Python 3. But that would make my timeline somewhat dependent on how much of a consensus there is, and how much clarity I could get before going forward.

In a separate RFE, Martin convinced me that we really want to kick as much of this off from type.init as we can, and that the problem with zero-argument super() currently not working when called from metaclass init methods should be treated as a bug in the way zero-argument super() is currently implemented.

That position makes a lot of sense to me (especially since it was backed up with a patch to fix the bug using a modifed implementation that's inspired by the way that setting qualname works), and is what makes it possible to assume we can just use the metaclass system to deal with this, rather than having to rely on modifications to build_class.

Even if PEP 422 never was officially tagged with "Approved" status in the PEP itself, our 2013 conversation with Guido made it sound like it was totally a done deal; if there was something other than PEP 487 that threw it off that track, I never saw it. So I'm understandably a little bit reluctant to start off implementing a new protocol that then two or three years from now will suddenly not be a done deal any more, with whatever I did being retroactively declared the wrong thing to do again.

That's entirely fair. The main thing that threw PEP 422 off track was that I received a lot of good requests for clarification in the previous round of PEP 422 discussions, and I wasn't really happy with the answers I was coming up with when I contemplated redrafting it, so it was easy to rationalise postponing further work on it. Martin's recent feedback then served to finally crystallise those doubts into "this isn't the right answer after all".

I suppose, though, that that my best option all in all is just to do whatever the heck seems best for porting, and worry about standardization later. If a member-notification protocol is standardized, I can always change DecoratorTools to use it later, after all, as long as the actual implementation mechanism inside DecoratorTools is opaque to its consumers. (i.e., if I don't actually expose the member-notification protocol directly)

(And, in retrospect, I could have, and probably should have, taken this approach from the get-go in 2012. It just seemed really, really important to you back then that I not do it.)

My apologies for that - while I don't actually recall what I was thinking when I said it, I suspect I was all fired up that PEP 422 was definitely the right answer, and hence thought I'd have an official solution in place for you in fairly short order. I should have let you know explicitly when I started having doubts about it, so you could reassess your porting options.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list