[Python-Dev] PEP 246, redux (original) (raw)

Phillip J. Eby pje at telecommunity.com
Mon Jan 10 18:43:44 CET 2005

At 03:42 PM 1/10/05 +0100, Alex Martelli wrote:

The fourth case above is subtle. A break of substitutability can occur when a subclass changes a method's signature, or restricts the domains accepted for a method's argument ("co-variance" on arguments types), or extends the co-domain to include return values which the base class may never produce ("contra-variance" on return types). While compliance based on class inheritance should be automatic, this proposal allows an object to signal that it is not compliant with a base class protocol.

-1 if this introduces a performance penalty to a wide range of adaptations (i.e. those using abstract base classes), just to support people who want to create deliberate Liskov violations. I personally don't think that we should pander to Liskov violators, especially since Guido seems to be saying that there will be some kind of interface objects available in future Pythons.

Just like any other special method in today's Python, conform is meant to be taken from the object's class, not from the object itself (for all objects, except instances of "classic classes" as long as we must still support the latter). This enables a possible 'tpconform' slot to be added to Python's type objects in the future, if desired.

One note here: Zope and PEAK sometimes use interfaces that a function or module may implement. PyProtocols' implementation does this by adding a conform object to the function's dictionary so that the function can conform to a particular signature. If and when conform becomes tp_conform, this may not be necessary any more, at least for functions, because there will probably be some way for an interface to tell if the function at least conforms to the appropriate signature. But for modules this will still be an issue.

I am not saying we shouldn't have a tp_conform; just suggesting that it may be appropriate for functions and modules (as well as classic classes) to have their tp_conform delegate back to self.dict['conform'] instead of a null implementation.

The object may return itself as the result of conform to indicate compliance. Alternatively, the object also has the option of returning a wrapper object compliant with the protocol. If the object knows it is not compliant although it belongs to a type which is a subclass of the protocol, then conform should raise a LiskovViolation exception (a subclass of AdaptationError). Finally, if the object cannot determine its compliance, it should return None to enable the remaining mechanisms. If conform raises any other exception, "adapt" just propagates it.

To enable the third case, when the protocol knows about the object, the protocol must have an adapt() method. This optional method takes two arguments: - self', the protocol requested_ _- obj', the object being adapted If the protocol finds the object to be compliant, it can return obj directly. Alternatively, the method may return a wrapper compliant with the protocol. If the protocol knows the object is not compliant although it belongs to a type which is a subclass of the protocol, then adapt should raise a LiskovViolation exception (a subclass of AdaptationError). Finally, when compliance cannot be determined, this method should return None to enable the remaining mechanisms. If adapt raises any other exception, "adapt" just propagates it. The fourth case, when the object's class is a sub-class of the protocol, is handled by the built-in adapt() function. Under normal circumstances, if "isinstance(object, protocol)" then adapt() returns the object directly. However, if the object is not substitutable, either the conform() or adapt() methods, as above mentioned, may raise an LiskovViolation (a subclass of AdaptationError) to prevent this default behavior.

I don't see the benefit of LiskovViolation, or of doing the exact type check vs. the loose check. What is the use case for these? Is it to allow subclasses to say, "Hey I'm not my superclass?" It's also a bit confusing to say that if the routines "raise any other exceptions" they're propagated. Are you saying that LiskovViolation is not propagated?

If none of the first four mechanisms worked, as a last-ditch attempt, 'adapt' falls back to checking a registry of adapter factories, indexed by the protocol and the type of `obj', to meet the fifth case. Adapter factories may be dynamically registered and removed from that registry to provide "third party adaptation" of objects and protocols that have no knowledge of each other, in a way that is not invasive to either the object or the protocols.

This should either be fleshed out to a concrete proposal, or dropped. There are many details that would need to be answered, such as whether "type" includes subtypes and whether it really means type or class. (Note that isinstance() now uses class, allowing proxy objects to lie about their class; the adaptation system should support this too, and both the Zope and PyProtocols interface systems and PyProtocols' generic functions support it.)

One other issue: it's not possible to have standalone interoperable PEP 246 implementations using a registry, unless there's a standardized place to put it, and a specification for how it gets there. Otherwise, if someone is using both say Zope and PEAK in the same application, they would have to take care to register adaptations in both places. This is actually a pretty minor issue since in practice both frameworks' interfaces handle adaptation, so there is no need for this extra registry in such cases.

Adaptation is NOT "casting". When object X itself does not conform to protocol Y, adapting X to Y means using some kind of wrapper object Z, which holds a reference to X, and implements whatever operation Y requires, mostly by delegating to X in appropriate ways. For example, if X is a string and Y is 'file', the proper way to adapt X to Y is to make a StringIO(X), NOT to call file(X) [which would try to open a file named by X].

Numeric types and protocols may need to be an exception to this "adaptation is not casting" mantra, however.

The issue isn't that adaptation isn't casting; why would casting a string to a file mean that you should open that filename? I don't think that "adaptation isn't casting" is enough to explain appropriate use of adaptation. For example, I think it's quite valid to adapt a filename to a factory for opening files, or a string to a "file designator". However, it doesn't make any sense (to me at least) to adapt from a file designator to a file, which IMO is the reason it's wrong to adapt from a string to a file in the way you suggest. However, casting doesn't come into it anywhere that I can see.

If I were going to say anything about that case, I'd say that adaptation should not be "lossy"; adapting from a designator to a file loses information like what mode the file should be opened in. (Similarly, I don't see adapting from float to int; if you want a cast to int, cast it.) Or to put it another way, adaptability should imply substitutability: a string may be used as a filename, a filename may be used to designate a file. But a filename cannot be used as a file; that makes no sense.

Reference Implementation and Test Cases

The following reference implementation does not deal with classic classes: it consider only new-style classes. If classic classes need to be supported, the additions should be pretty clear, though a bit messy (x.class vs type(x), getting boundmethods directly from the object rather than from the type, and so on).

Please base a reference implementation off of either Zope or PyProtocols' field-tested implementations which deal correctly with class vs. type(), and can detect whether they're calling a conform or adapt at the wrong metaclass level, etc. Then, if there is a reasonable use case for LiskovViolation and the new type checking rules that justifies adding them, let's do so.

Transitivity of adaptation is in fact somewhat controversial, as is the relationship (if any) between adaptation and inheritance.

The issue is simply this: what is substitutability? If you say that interface B is substitutable for A, and C is substitutable for B, then C must be substitutable for A, or we have inadequately defined "substitutability".

If adaptation is intended to denote substitutability, then there can be absolutely no question that it is transitive, or else it is not possible to have any meaning for interface inheritance!

Thus, the controversies are: 1) whether adaptation should be required to indicate substitutability (and I think that your own presentation of the string->file example supports this), and 2) whether the adaptation system should automatically provide an A when provided with a C. Existing implementations of interfaces for Python all do this where interface C is a subclass of A. However, they differ as to whether all adaptation should indicate substitutability. The Zope and Twisted designers believe that adaptation should not be required to imply substitutability, and that only interface and implementation inheritance imply substitutability. (Although, as you point out, the latter is not always the case.)

PyProtocols OTOH believes that all adaptation must imply substitutability; non-substitutable adaptation or inheritance is a design error: "adaptation abuse", if you will. So, in the PyProtocols view, it would never make sense to define an adaptation from float or decimal to integer that would permit loss of precision. If you did define such an adaptation, it must refuse to adapt a float or decimal with a fractional part, since the number would no longer be substitutable if data loss occurred.

Of course, this is a separate issue from automatic transitive adaptation, in the sense that even if you agree that adaptation must imply substitutability, you can still disagree as to whether automatically locating a multi-step adaptation is desirable enough to be worth implementing. However, if substitutability is guaranteed, then such multi-step adaptation cannot result in anything "controversial" occurring.

The latter would not be controversial if we knew that inheritance always implies Liskov substitutability, which, unfortunately we don't. If some special form, such as the interfaces proposed in [4], could indeed ensure Liskov substitutability, then for that kind of inheritance, only, we could perhaps assert that if X conforms to Y and Y inherits from Z then X conforms to Z... but only if substitutability was taken in a very strong sense to include semantics and pragmatics, which seems doubtful.

As a practical matter, all of the existing interface systems (Zope, PyProtocols, and even the defunct Twisted implementation) treat interface inheritance as guaranteeing substitutability for the base interface, and do so transitively.

However, it seems to me to be a common programming error among people new to interfaces to inherit from an interface when they intend to require the base interface's functionality, rather than offer the base interface's functionality. It may be worthwhile to address this issue in the design of "standard" interfaces for Python.

This educational issue regarding substitutability is I believe inherent to the concept of interfaces, however, and does not go away simply by making non-inheritance adaptation non-transitive in the implementation. It may, however, make it take longer for people to encounter the issue, thereby slowing their learning process. ;)

Backwards Compatibility

There should be no problem with backwards compatibility unless someone had used the special names conform or adapt in other ways, but this seems unlikely, and, in any case, user code should never use special names for non-standard purposes.

Production implementations of the old version of PEP 246 exist, so the changes in semantics you've proposed may introduce backward compatibility issues. More specifically, some field code may not work correctly with your proposed reference implementation, in the sense that code that worked with Zope or PyProtocols before, may not work with the reference implementation's adapt(), resulting in failure of adaptation where success occurred before, or in exceptions raised where no exception was raised before.

More information about the Python-Dev mailing list