[Python-Dev] Status of PEP 484 and the typing module (original) (raw)

Guido van Rossum guido at python.org
Thu May 21 22:27:50 CEST 2015

Things are looking up. I think we're down to a very small number of issues where we still disagree -- hopefully you'll allow me some leeway. :-)

On Thu, May 21, 2015 at 8:45 AM, Mark Shannon <mark at hotpy.org> wrote:

On 21/05/15 16:01, Guido van Rossum wrote: Hi Mark,

We're down to the last few items here. I'm CC'ing python-dev so folks can see how close we are. I'll answer point by point. On Thu, May 21, 2015 at 6:24 AM, Mark Shannon <mark at hotpy.org_ _<mailto:mark at hotpy.org>> wrote: Hi, The PEP itself is looking fairly good.

I hope you'll accept it at least provisionally so we can iterate over the finer points while a prototype of typing.py in in beta 1. However, I don't think that typing.py is ready yet, for a number of reasons: 1. As I've said before, there needs to be a distinction between classes and types. They is no need for Any, Generic, Generic's subtypes, or Union to subclass builtins.type. I strongly disagree. They can appear in many positions where real classes are acceptable, in particular annotations can have classes (e.g. int) or types (e.g. Union[int, str]). Why does this mean that they have to be classes? Annotations can be any object.

I want to encourage users to think about annotations as types, and for most users the distinction between type and class is too subtle, so a simpler rule is to say they are classes. This works out nicely when the annotations are simple types such as 'int' or 'str' or user-defined classes (e.g. 'Employee').

It might to help to think, not in terms of types being classes, but classes being shorthand for the nominal type for that class (from the point of view of the checker and type geeks) So when the checker sees 'int' it treats it as Type(int).

I'm fine with that being the formal interpretation (except that I don't want to introduce a function named Type()). But it's too subtle for most users.

Subtyping is distinct from subclassing; Type(int) <: Union[Type(int), Type(str)] has no parallel in subclassing. There is no class that corresponds to a Union, Any or a Generic.

Again, for most people te distinction is too subtle. People expect to be able to play around with things interactively. I think it will be helpful if they can experiment with the objects exported by typing too: experimenting with things like isinstance(42, Union[int, str]) or issubclass(Any, Employee) and issubclass(Employee, Any) is a useful thing to explore how these things work (always with the caveat that when Any is involved, issubclass is not transitive). Of course it won't work when they advance to type variables -- at that point you just have to understand the theory and switch from using the interactive interpreter to writing small test programs and seeing how mypy (or some other checker) responds to them.

In order to support the class C(ParameterType[T]): pass

I presume you mean class C(Generic[T])?

syntax, parametric types do indeed need to be classes, but Python has multiple inheritance, so thats not a problem: class ParameterType(type, Type): ... Otherwise typing.Types shouldn't be builtin.types and vice versa.

There's one thing here that Jukka has convinced me of. While I really want Union[...] to act like a class (though not subclassable!), plain Union (without the [...]) needn't. The same is true for Callable and Tuple without [...]. I've filed https://github.com/ambv/typehinting/issues/133 for this. I'm not sure how much work it will be to fix this but I don't think it absolutely needs to be done in beta 1 -- there's not much you can do with them anyway.

I think a lot of this issues on the tracker would not have been issues had the distinction been more clearly enforced.

Playing around with typing.py, it has also become clear to me that it is also important to distinguish type constructors from types. What do I mean by a type constructor? A type constructor makes types. "List" is an example of a type constructor. It constructs types such as List[T] and List[int]. Saying that something is a List (as opposed to a list) should be rejected.

The PEP actually says that plain List (etc.) is equivalent to List[Any]. (Well, at least that's the intention; it's implied by the section about the equivalence between Node() and NodeAny. Perhaps we should change that. Using 'List', rather than 'list' or 'List[Any]' suggests an error, or misunderstanding, to me. Is there a use case where 'List' is needed, and 'list' will not suffice? I'm assuming that the type checker knows that 'list' is a MutableSequence.

I think it's easier if we ask people to always write 'List' rather than 'list' when they are talking about types, and 'List[Any]' will probably be a popular type (lots of people don't want to think about exactly what the item type is, but they are sure that the container is a list).

There's also an argument from consistency with the collection ABCs. As you know, typing defines a bunch of types that act as "stand ins" for the corresponding ABCs defined in collections.abc (Iterable, Sequence, Sized, etc.). The intention here is that anywhere one of the collection ABCs is valid it should be okay to use the corresponding class imported from typing -- so that if you have code that currently uses "from collections.abc import Sequence, Mapping" you can just replace that with "from typing import Sequence, Mapping" and your code will still work. (You can then iterate at leisure on parametrizing the types.)

So we can use e.g. Sequence as a base class and it means the same as Sequence[Any]. Given this rule, it would be somewhat surprising if you couldn't use List but were forced to write List[Any] in other places. (Neither Sequence[Any] nor List[Any] can be instantiated so that's not a concern.)

2. Usability of typing as it stands: Let's try to make a class that implements a mutable mapping. >>> import typing as tp #Make some variables. >>> T = tp.TypeVar('T') >>> K = tp.TypeVar('K') >>> V = tp.TypeVar('V') #Then make our class: >>> class MM(tp.MutableMapping): pass ... #Oh that worked, but it shouldn't. MutableMapping is a type constructor.

It means MutableMapping[Any]. #Let's make one >>> MM() Traceback (most recent call last): File "", line 1, in File "/home/mark/repositories/typehinting/prototyping/typing.py", line 1095, in new if gorg(c) is Generic: File "/home/mark/repositories/typehinting/prototyping/typing.py", line 887, in gorg while a.origin is not None: AttributeError: type object 'Sized' has no attribute 'origin' # ??? Sorry, that's a bug I introduced in literally the last change to typing.py. I will fix it. The expected behavior is TypeError: Can't instantiate abstract class MM with abstract methods len #Well let's try using type variables. class MM2(tp.MutableMapping[K, V]): pass ... >>> MM2() Traceback (most recent call last): File "", line 1, in File "/home/mark/repositories/typehinting/prototyping/typing.py", line 1095, in new if gorg(c) is Generic: File "/home/mark/repositories/typehinting/prototyping/typing.py", line 887, in gorg while a.origin is not None: AttributeError: type object 'Sized' has no attribute 'origin' Ditto, and sorry. No need to apologise, I'm just a bit worried about how easy it was for me to expose this sort of bug.

Well, I'm just glad you exposed it so soon after I introduced it. :-)

At this point, we have to resort to using 'Dict', which forces us to subclass 'dict' which may not be what we want as it may cause metaclass conflicts. 3. Memory consumption is also a worry. There is no caching, which means every time I use "List[int]" as an annotation, a new class object is created. Each class may only be a few KB, but collectively this could easily add up to several MBs. This should be easy to fix. I can work on this after the beta-1 release. Until then, type aliases can be used to avoid redundant type creation (and often they are clearer anyway :-). Sure.

OK. Tracking this in https://github.com/ambv/typehinting/issues/130

[...]

-- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20150521/d9db4af5/attachment-0001.html>

More information about the Python-Dev mailing list