[Python-3000] Updated and simplified PEP 3141: A Type Hierarchy for Numbers (original) (raw)

Guido van Rossum guido at python.org
Thu May 17 19:53:42 CEST 2007

On 5/17/07, Talin <talin at acm.org> wrote:

Lets therefore assume that the numeric ABCs will use this new inheritance mechanism, avoiding the problem of taking an immature class hierarchy and setting it in stone. The PEPs in this class would then no longer need to have this privileged status; They could be replaced and changed at will.

Assuming that this is true, the question then becomes whether these classes should be treated like any other standard library submission. In other words, shouldn't this PEP be implemented as a separate module, and have to prove itself 'in the wild' before being adopted into the stdlib? Does this PEP even need to be a PEP at all, or can it just be a 3rd-party library that is eventually adopted into Python?

No; I think there's a lot of synergy to be had by making it a standard library module. For example, the Complex, Real and Integer types provide a common ground for the built-in types and the types implemented in numpy. Assuming (some form of) PEP 3124 is accepted, it would be a shame if we had to specialize GFs on concrete types like int or float. If we could encourage the habit right from the start to use the abstract classes in such positions, then numpy integration would be much easier.

Now, I could see adopting an untried library embodying untested ideas into the stdlib if there was a crying need for the features of such a library, and those needs were clearly being unfulfilled. However, I am not certain that this is the case here.

The ideas here are hardly untested; the proposed hierarchy is taught in high school math if not before, and many other languages use it (e.g. Scheme's numeric tower, referenced in the PEP). Some implementation details are untested, but I doubt that the general idea sparks much controversy (it hasn't so far).

At the very least, I think it should be stated in the PEP whether or not the ABCs defined here are going to be using traditional or dynamic inheritance.

Dynamic inheritance for sure.

If it is the latter, and we decide that this PEP is going to be part of the stdlib, then I propose the following library organization:

import abc # Imports the basic ABC mechanics import abc.collections # MutableSequence and such import abc.math # The number hierarchy ... and so on

I don't like the idea of creating a ghetto for ABCs. Just like the ABCs for use with I/O are defined in the io module (read PEP 3161 and notice that it already uses a thinly disguised form of ABCs), the ABCs for collections should be in the existing collections module. I'm not sure where to place the numeric ABCs, but I'd rather have a top-level numbers module.

Now, there is another issue that needs to be dicussed.

The classes in the PEP appear to be written with lots of mixin methods, such as rsub and abs and such. Unfortunately, the current proposed method for dynamic inheritance does not allow for methods or properties to be inherited from the 'virtual' base class. Which means that all of the various methods defined in this PEP are utterly meaningless other than as documentation - except in the case of a new user-created class of numbers which inherit from these ABCs using traditional inheritance, which is not something that I expect to happen very often at all. For virtually all practical uses, the elaborate methods defined in this PEP will be unused and inaccessible.

And yet they are designed for the benefit of new numeric type implementations so that mixed-mode arithmetic involving two different 3rd party implementations can be defined soundly. If this is deemed too controversial or unwieldy and unnecessary I'd be okay with dropping it, though I'm not sure that it hurts. There are some problems with specifying it so that all the cases work right, however. In any case we could have separate mix-in classes for this purpose -- while ABCs can be dual-purpose (both specification and mix-in), there's no rule that says the must be.

The problem Jeffrey and I were trying to solve with this part of the spec is what should happen if you have two independently developed 3rd party types, e.g. MyReal and YourReal, both implementing the Real ABC. Obviously we want MyReal("3.5") + YourReal("4.5") to return an object for which isinstance(x, Real) holds and whose value is close to float("8.0"). But this is not so easy. Typically, classes like this have methods like these::

class MyReal: def add(self, other): if not isinstance(other, MyReal): return NotImplemented return MyReal(...) def radd(self, other): if not isinstance(other, MyReal): return NotImplemented return MyReal(...)

but this doesn't support mixed-mode arithmetic at all.

(Reminder of the underlying mechanism: for a+b, first a.add(b) is tried; if that returns NotImplemented, b.radd(a) is tried; if that also returns NotImplemented, the Python VM raises TypeError. Exceptions raised at any stage cause the remaining steps to be abandoned, so if e.g. add raises TypeError, radd is never tried. This is the crux of the problem we're trying to solve.)

Supporting mixed-mode arithmetic with known other types like float is easy: just add either

if isinstance(other, float): return self + MyReal(other)

or

if isinstance(other, float): return float(self) + other

to the top of each method. But that still doesn't support MyReal() + YourReal(). For that to work, at least one of the two classes has to blindly attempt to cast the other argument to a known class. For example, instead of returning NotImplemented, radd (which knows it is being called as a last resort) could return self+float(other), or float(self)+float(other), under the assumption that all 3rd party Real types can be converted to the built-in float type with only moderate loss.

Unfortunately, there are a lot of cases to consider here. E.g. we could be adding MyComplex() to YourReal(), and then the float cast in radd would be a disaster (since MyComplex() can't be cast to float, only to complex). We are trying to make things easier for 3rd parties that do want to use the ABC as a mix-in, by asking them to call super.[r]add(self, other) whenever the other argument is not something they specifically recognize.

I'm pretty sure that we haven't gotten the logic for this quite right yet. I'm only moderately sure that we can get it right. But in any case, please don't let this distract you from the specification part of the numeric ABCs.

This really highlights what I think is a problem with dynamic inheritance, and I think that this inconsistency between traditional and dynamic inheritance will eventually come back to haunt us. It has always been the case in the past that for every property of class B, if isinstance(A, B) == True, then A also has that property, either inherited from B, or overridden in A. The fact that this invariant will no longer hold true is a problem in my opinion.

Actually there is a school of thought (which used to prevail amongst Zopistas, I don't know if they've been cured yet) that class inheritance was purely for implementation inheritance, and that a subclass was allowed to reverse policies set by the base class. While I don't endorse this as a general rule, I don't see how (with a sufficiently broad definition of "property") your invariant can be assumed even for traditional inheritance. E.g. a base class may be hashable or or immutable but the subclass may not be (it's trivial to create mutable subclasses of int, str or tuple).

I realize that there isn't currently a solution to efficiently allow inheritance of properties via dynamic inheritance. As a software engineer, however, I generally feel that if a feature is unreliable, then it shouldn't be used at all.

That sounds like a verdict against all dynamic properties of the language.

So if I were designing a class hierarchy of ABCs, I would probably make a rule for myself not to define any properties or methods in the ABCs at all, and to only use ABCs for type testing via 'isinstance'.

And that is a fine policy to hold yourself to.

In other words, if I were writing this PEP,

Hey, you are! Or do you want your name taken off? At the very least I think the rationale (all your words) needs some ervision in the light of Benji York's comments in another thread.

all of those special methods would be omitted, simply because as a writer of a subclass I couldn't rely on being able to use them.

As the author of a subclass, you control the choice between real inheritance via inclusion in bases or pseudo-inheritance via register(). So I don't see your point here.

The only alternative that I can see is to not use dynamic inheritance at all, and instead have the number classes inherit from these ABCs using the traditional mechanism. But that brings up all the problems of immaturity and requiring them to be built-in that I brought up earlier.

That's off the table already.

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-3000 mailing list