[Python-3000] ABC's, Roles, etc (original) (raw)

Phillip J. Eby pje at telecommunity.com
Wed May 9 03:57:37 CEST 2007

At 03:52 PM 5/8/2007 -0600, Jeff Shell wrote:

I have a lengthy post that dissects a major issue that I have with ABCs and the Interface definition that I saw in PEP 3124:: it all seems rigidly class and class-instance based.

Hi Jeff; I read your post a few days ago, but your blog doesn't support comments, so I've been "getting around to" writing a counterpoint on my blog. But, now I can do it here instead. :)

The cardinal sin I saw in the Interface definition in PEP 3124 (at least, at the time I last viewed it) was the inclusion of 'self' in a method spec.

That's because you're confusing a generic function and a "method spec". PEP 3124 interfaces are not specifications; they're namespaces for generic functions. They are much closer in nature to ABCs than they are zope.interface-style Interfaces. The principal thing they have in common with zope.interface (aside from the name) is the support for "IFoo(ob)"-style adaptation.

Very few of zope.interface's design goals are shared by PEP 3124. Notably, they are not particularly good for type checking or verification. In PEP 3124, for example, IFoo(ob) always returns an object with the specified attributes; the only way to know whether they are actually implemented is to try using them.

Notice that this is diametrically opposed to what zope.interface wants to do in such situations -- which is why the PEP makes such a big deal about it being possible to use zope.interface instead.

That is, my own observation is that different frameworks sometimes need different kinds of interfaces. For example, someone might create a framework that wants to verify preconditions and postconditions of methods in an interface, rather than merely specifying their names and arguments! Using zope.interface as an exclusive basis for interface definition and type annotations would block innovation in this area.

PEP 3124 interfaces are therefore explicitly intended to be merely one possible kind of interface, rather than a be-all end-all interface system. They have many differences from zope.interface, which, depending on your goals, may be a plus or minus. But you certainly aren't obligated to use them. PEP 3124 merely proposes a framework for how to use interfaces for method overloading, generic functions, and AOP.

From the PEP itself:

"""For example, it should be possible to use a zope.interface interface object to specify the desired type of a function argument, as long as the zope.interface package registered itself correctly (or a third party did the registration).

In this way, the proposed API simply offers a uniform way of accessing the functionality within its scope, rather than prescribing a single implementation to be used for all libraries, frameworks, and applications."""

... 'self' is an internal detail of class-instance implementations.

Again - this is because you're assuming the purpose of a PEP 3124 interface is to specify an interface, when in fact it's much more like an ABC, which may also implement the interface. The specification and implementation are intentionally unified here.

Of course, again, you will be able to use zope.interfaces as argument annotations to @overload-ed functions and methods, which is the point of the PEP. Its "Interface" class is merely a suggested default, leaving the fancier tricks to established packages, in the same way that its generic function implementation will not do everything that RuleDispatch or PEAK-Rules can do. It's supposed to be a core framework for such add-on packages, not a replacement for them.

It seems to me that Abstract Base Classes and even PEP 3124 are primarily focused on classes. But in Python, "everything is an object", but not everything is class-based. ... The rest of this post focuses on what zope.interface already provides - a system for specifying behavior and declaring support at both the class and object level - and 'object' really means 'object', which includes modules.

Right -- and neither "specifying behavior" nor "declaring support" are goals of PEP 3124; they're entirely out of its scope. The Interface object's purpose is to support uniform access to, and implementation of, individual operations. These are somewhat parallel concepts, but very different in thrust; zope.interface is LBYL, while PEP 3124 is EAFP all the way.

As a result, PEP 3124 chooses to punt on the issue of individual objects. It's quite possible within the framework to allow instance-level checks during dispatching, but it's not going to be in the default engine (which is based on type-tuple dispatching; see Guido's Py3K overloading prototype).

zope.interface (and zope.component, IIRC) pay a high price in complexity for allowing interfaces to be per-instance rather than type-defined. I implemented the same feature in PyProtocols, but over the years I rarely found it useful. My understanding of its usefulness in Zope is that it:

  1. supports specification and testing of module-level Zope APIs
  2. allows views and other wrapping operations to be selected on a dynamic basis

Since #1 falls outside of PEP 3124's goals (i.e., it's not about specification or testing), that leaves use case #2. In my experience, it has been more than sufficient to simply give these object some other interface, such as an IViewTags interface with a method to query these dynamic "tag" interfaces. In other words, my experience and opinion supports the view that use case #2 is actually a coincidental abuse of interfaces for convenience, rather than the "one obvious way" to handle the use case.

To put it another way, if you define getView() as a generic function, you can always define a dynamic implementation of it for any type that you wish to have dynamic view selection capability. Then, only those cases that require a complex solution have to pay for the complexity.

So, that's my rationale for why PEP 3124 doesn't provide any instance-based features out of the box; outside of API specs for singleton objects, the need for them is mostly an illusion created by Zope 3's dynamic view selection hack.

My main focus is on determining what Abstract Base Classes and/or PEP 3124's Interfaces do better than zope.interface (if anyone else is familiar with that package).

I at least am quite familiar with it, having helped to define some of its terminology and API, as well as being the original author of its class-decorator emulation for Python versions 2.2 and up. :) I also argued for its adoption of PEP 246, and wrote PyProtocols to unify Twisted and Zope interfaces in a PEP 246-based adaptation framework.

And what PEP 3124 does much better than zope.interface or even PyProtocols is:

  1. Adaptation, especially incomplete adaptation. You can implement only the methods that are actually needed for your use case. If the interface includes generic implementations that are defined in terms of other methods in the interface, you need not reimplement them. (Note: I'm well aware that here my definition of "better" would be considered "worse" by Jim Fulton, since zope.interface is LBYL-oriented. However, for many users and use cases, EAFP is better, even if it's not for Zope.)

  2. Interface recombination. AFAIK, zope.interface doesn't support subset interfaces like PyProtocols does. Neither zope.interface nor PyProtocols support method renaming, where two interfaces have a method with the same specification but different method names.

  3. Low mental overhead. PEP 3124 doesn't even need interfaces; simple use cases can just use overloaded functions and be on about their business. Use cases that would require an interface and half a dozen adapter classes in zope.interface can be met by simply creating an overloaded function and adding methods. And the resulting code reads like code in other languages that support overloading or generic functions, rather than reading like Java.

In my blog post, I also show a dynamically constructed object providing an interface's specified behavior. An instance of an empty class is made, and then methods and other supporting attributes are attached to this specific instance only. Real world examples of this include Zope 2, where a folder may have "Python Scripts" or other callable members that, in effect, make for a totally custom object. It can also provide this same behavior (in fact, I was able to take advantage of this on some old old old Zope 2 projects that started in the web environment and transitioned to regular Python modules/classes).

And how often does this happen outside of Zope? As I said, I rarely found it to be the case anywhere else. I replicated the ability in PyProtocols because I was biased by my prior Zope experience, but once I got outside of Zope it almost entirely ceased to be useful.

Meanwhile, as I said, PEP 3124 is not closed to extension. It's specifically intended that zope.interface (and any other interface packages that might arise in future) should be able to play as first-class citizens in the proposed API. However, depending on the specific features desired, those packages might have some additional integration work to do.

(Note, by the way, that zope.interface is explicitly mentioned three times in the PEP, as an example of how other interface types should be able to be used for overloading, as long as they register appropriate methods with the provided framework.)

More information about the Python-3000 mailing list