[Python-Dev] PEP 484 (Type Hints) -- the next draft is here (original) (raw)

Guido van Rossum guido at python.org
Fri Apr 17 23:58:25 CEST 2015

The PEP 484 authors (Jukka, Łukasz and I) have a new draft ready. This time we're posting to python-dev.

While we don't have all details totally sorted out, it's a lot closer. We have a BDFL-Delegate (Mark Shannon), and we've worked out a variety of issues at PyCon. Our hope remains that we'll get the typing.py module added to CPython 3.5 before beta 1 goes out (May 24).

Below is a list of key changes since the draft posted to python-dev on March 20 (for more details see https://github.com/ambv/typehinting/commits/master/pep-0484.txt) and the full text of the PEP. The new draft is also in the peps repo ( https://hg.python.org/peps/file/tip/pep-0484.txt) and will soon be on python.org (https://www.python.org/dev/peps/pep-0484/ -- give it 10-30 minutes).

As always, between draft postings the PEP text is updated frequently by the authors in a dedicated GitHub repo (https://github.com/ambv/typehinting), and many detailed discussions are found in the issue tracker there ( https://github.com/ambv/typehinting/issues). The typing.py module also lives in that repo ( https://github.com/ambv/typehinting/tree/master/prototyping).

Key changes since 20-Mar-2015 draft

Generics:

Type specifications:

Specifics:

Stubs:

Process:

Other:

Full PEP text

PEP: 484 Title: Type Hints Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Guido van Rossum <guido at python.org>, Jukka Lehtosalo < jukka.lehtosalo at iki.fi>, Łukasz Langa <lukasz at langa.pl> BDFL-delegate: Mark Shannon Discussions-To: Python-Dev <python-dev at python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 29-Sep-2014 Post-History: 16-Jan-2015,20-Mar-2015,17-Apr-2015 Resolution:

Abstract

This PEP introduces a standard syntax for type hints using annotations (PEP 3107) on function definitions. For example, here is a simple function whose argument and return type are declared in the annotations::

def greeting(name: str) -> str: return 'Hello ' + name

While these annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily. Essentially, such a type checker acts as a very powerful linter.

The proposal is strongly inspired by mypy [mypy]_. For example, the type "sequence of integers" can be written as Sequence[int]. The square brackets mean that no new syntax needs to be added to the language. The example here uses a custom class Sequence, imported from a pure-Python module typing. The Sequence[int] notation works by implementing __getitem__() in the metaclass.

The type system supports unions, generic types, and a special type named Any which is consistent with (i.e. assignable to and from) all types. This latter feature is taken from the idea of gradual typing. Gradual typing and the full type system are explained in PEP 483.

Other approaches from which we have borrowed or to which ours can be compared and contrasted are described in PEP 482.

Rationale and Goals

PEP 3107 added support for arbitrary annotations on parts of a function definition. Although no meaning was assigned to annotations then, there has always been an implicit goal to use them for type hinting, which is listed as the first possible use case in said PEP.

This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and performance optimizations utilizing type information.

Of these goals, static analysis is the most important. This includes support for off-line type checkers such as mypy, as well as providing a standard notation that can be used by IDEs for code completion and refactoring.

Non-goals

While the proposed typing module will contain some building blocks for runtime type checking -- in particular a useful isinstance() implementation -- third party packages would have to be developed to implement specific runtime type checking functionality, for example using decorators or metaclasses. Using type hints for performance optimizations is left as an exercise for the reader.

It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention.

What is checked?

Any function (or method -- for brevity we won't be repeating this) with at least one argument or return annotation is checked, unless type checking is disabled by the @no_type_check decorator or a # type: ignore comment (see below).

A checked function should have annotations for all its arguments and its return type, with the exception that the self argument of a method should not be annotated; it is assumed to have the type of the containing class. Notably, the return type of __init__ should be annotated with -> None.

The body of a checked function is checked for consistency with the given annotations. The annotations are also used to check correctness of calls appearing in other checked functions.

Functions without any annotations (or whose checking is disabled) are assumed to have type Any when they are referenced in checked functions, and this should completely silence complaints from the checker regarding those references (although a checker may still request that a type be specified using a cast or a # type: comment if a more specific type than Any is needed for analysis of subsequent code).

A type checker should understand decorators; this may require annotations on decorator definitions. In particular, a type checker should understand the built-in decorators @property, @staticmethod and @classmethod. The first argument of a class method should not be annotated; it is assumed to be a subclass of the defining class.

Type Definition Syntax

The syntax leverages PEP 3107-style annotations with a number of extensions described in sections below. In its basic form, type hinting is used by filling function annotation slots with classes::

def greeting(name: str) -> str: return 'Hello ' + name

This states that the expected type of the name argument is str. Analogically, the expected return type is str.

Expressions whose type is a subtype of a specific argument type are also accepted for that argument.

Acceptable type hints

Type hints may be built-in classes (including those defined in standard library or third-party extension modules), abstract base classes, types available in the types module, and user-defined classes (including those defined in the standard library or third-party modules). Annotations for built-in classes (and other classes at the discretion of the developer) may be placed in stub files (see below).

Annotations must be valid expressions that evaluate without raising exceptions at the time the function is defined (but see below for forward references).

The needs of static analysis require that annotations must be simple enough to be interpreted by static analysis tools. In particular, dynamically computed types are not acceptable. (This is an intentionally somewhat vague requirement, specific inclusions and exclusions may be added to future versions of this PEP as warranted by the discussion.)

In addition to the above, the following special constructs defined below may be used: None, Any, Union, Tuple, Callable, all ABCs and stand-ins for concrete classes exported from typing (e.g. Sequence and Dict), type variables, and type aliases.

All newly introducedw names used to support features described in following sections (such as Any and Union) are available in the typing module.

Using None

When used in a type hint, the expression None is considered equivalent to type(None).

Type aliases

Type aliases are defined by simple variable assignments::

Url = str

def retry(url: Url, retry_count: int) -> None: ...

Note that we recommend capitalizing alias names, since they represent user-defined types, which (like user-defined classes) are typically spelled that way.

Type aliases may be as complex as type hints in annotations -- anything that is acceptable as a type hint is acceptable in a type alias::

from typing import TypeVar, Iterable, Tuple

T = TypeVar('T', int, float, complex)
Vector = Iterable[Tuple[T, T]]

def inproduct(v: Vector) -> T:
    return sum(x*y for x, y in v)

This is equivalent to::

from typing import TypeVar, Iterable, Tuple

T = TypeVar('T', int, float, complex)

def inproduct(v: Iterable[Tuple[T, T]]) -> T:
    return sum(x*y for x, y in v)

Callable

Frameworks expecting callback functions of specific signatures might be type hinted using Callable[[Arg1Type, Arg2Type], ReturnType]. Examples::

from typing import Callable

def feeder(get_next_item: Callable[[], str]) -> None: # Body

def async_query(on_success: Callable[[int], None], on_error: Callable[[int, Exception], None]) -> None: # Body

It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis (three dots) for the list of arguments::

def partial(func: Callable[..., str], *args) -> Callable[..., str]: # Body

Note that there are no square brackets around the ellipsis. The arguments of the callback are completely unconstrained in this case (and keyword arguments are acceptable).

Since using callbacks with keyword arguments is not perceived as a common use case, there is currently no support for specifying keyword arguments with Callable. Similarly, there is no support for specifying callback signatures with a variable number of argument of a specific type.

Generics

Since type information about objects kept in containers cannot be statically inferred in a generic way, abstract base classes have been extended to support subscription to denote expected types for container elements. Example::

from typing import Mapping, Set

def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]) -> None: ...

Generics can be parametrized by using a new factory available in typing called TypeVar. Example::

from typing import Sequence, TypeVar

T = TypeVar('T') # Declare type variable

def first(l: Sequence[T]) -> T: # Generic function return l[0]

In this case the contract is that the returned value is consistent with the elements held by the collection.

TypeVar supports constraining parametric types to a fixed set of possible types. For example, we can define a type variable that ranges over just str and bytes. By default, a type variable ranges over all possible types. Example of constraining a type variable::

from typing import TypeVar

AnyStr = TypeVar('AnyStr', str, bytes)

def concat(x: AnyStr, y: AnyStr) -> AnyStr: return x + y

The function concat can be called with either two str arguments or two bytes arguments, but not with a mix of str and bytes arguments.

Note that subtypes of types constrained by a type variable are treated as their respective explicitly listed base types in the context of the type variable. Consider this example::

class MyStr(str): ...

x = concat(MyStr('apple'), MyStr('pie'))

The call is valid but the type variable AnyStr will be set to str and not MyStr. In effect, the inferred type of the return value assigned to x will also be str.

Additionally, Any is a valid value for every type variable. Consider the following::

def count_truthy(elements: List[Any]) -> int: return sum(1 for elem in elements if element)

This is equivalent to omitting the generic notation and just saying elements: List.

User-defined generic types

You can include a Generic base class to define a user-defined class as generic. Example::

from typing import TypeVar, Generic

T = TypeVar('T')

class LoggedVar(Generic[T]): def init(self, value: T, name: str, logger: Logger) -> None: self.name = name self.logger = logger self.value = value

  def set(self, new: T) -> None:
      self.log('Set ' + repr(self.value))
      self.value = new

  def get(self) -> T:
      self.log('Get ' + repr(self.value))
      return self.value

  def log(self, message: str) -> None:
      self.logger.info('{}: {}'.format(self.name message))

Generic[T] as a base class defines that the class LoggedVar takes a single type parameter T. This also makes T valid as a type within the class body.

The Generic base class uses a metaclass that defines __getitem__ so that LoggedVar[t] is valid as a type::

from typing import Iterable

def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None: for var in vars: var.set(0)

A generic type can have any number of type variables, and type variables may be constrained. This is valid::

from typing import TypeVar, Generic ...

T = TypeVar('T') S = TypeVar('S')

class Pair(Generic[T, S]): ...

Each type variable argument to Generic must be distinct. This is thus invalid::

from typing import TypeVar, Generic ...

T = TypeVar('T')

class Pair(Generic[T, T]): # INVALID ...

You can use multiple inheritance with Generic::

from typing import TypeVar, Generic, Sized

T = TypeVar('T')

class LinkedList(Sized, Generic[T]): ...

Arbitrary generic types as base classes

Generic[T] is only valid as a base class -- it's not a proper type. However, user-defined generic types such as LinkedList[T] from the above example and built-in generic types and ABCs such as List[T] and Iterable[T] are valid both as types and as base classes. For example, we can define a subclass of Dict that specializes type arguments::

from typing import Dict, List, Optional

class Node: ...

class SymbolTable(Dict[str, List[Node]]): def push(self, name: str, node: Node) -> None: self.setdefault(name, []).append(node)

  def pop(self, name: str) -> Node:
      return self[name].pop()

  def lookup(self, name: str) -> Optional[Node]:
      nodes = self.get(name)
      if nodes:
          return nodes[-1]
      return None

SymbolTable is a subclass of dict and a subtype of Dict[str, List[Node]].

If a generic base class has a type variable as a type argument, this makes the defined class generic. For example, we can define a generic LinkedList class that is iterable and a container::

from typing import TypeVar, Iterable, Container

T = TypeVar('T')

class LinkedList(Iterable[T], Container[T]): ...

Now LinkedList[int] is a valid type. Note that we can use T multiple times in the base class list, as long as we don't use the same type variable T multiple times within Generic[...].

Abstract generic types

The metaclass used by Generic is a subclass of abc.ABCMeta. A generic class can be an ABC by including abstract methods or properties, and generic classes can also have ABCs as base classes without a metaclass conflict.

Forward references

When a type hint contains names that have not been defined yet, that definition may be expressed as a string literal, to be resolved later.

A situation where this occurs commonly is the definition of a container class, where the class being defined occurs in the signature of some of the methods. For example, the following code (the start of a simple binary tree implementation) does not work::

class Tree: def init(self, left: Tree, right: Tree): self.left = left self.right = right

To address this, we write::

class Tree: def init(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right

The string literal should contain a valid Python expression (i.e., compile(lit, '', 'expr') should be a valid code object) and it should evaluate without errors once the module has been fully loaded. The local and global namespace in which it is evaluated should be the same namespaces in which default arguments to the same function would be evaluated.

Moreover, the expression should be parseable as a valid type hint, i.e., it is constrained by the rules from the section Acceptable type hints_ above.

It is allowable to use string literals as part of a type hint, for example::

class Tree:
    ...
    def leaves(self) -> List['Tree']:
        ...

Union types

Since accepting a small, limited set of expected types for a single argument is common, there is a new special factory called Union. Example::

from typing import Union

def handle_employees(e: Union[Employee, Sequence[Employee]]) -> None: if isinstance(e, Employee): e = [e] ...

A type factored by Union[T1, T2, ...] responds True to issubclass checks for T1 and any of its subclasses, T2 and any of its subclasses, and so on.

One common case of union types are optional types. By default, None is an invalid value for any type, unless a default value of None has been provided in the function definition. Examples::

def handle_employee(e: Union[Employee, None]) -> None: ...

As a shorthand for Union[T1, None] you can write Optional[T1]; for example, the above is equivalent to::

from typing import Optional

def handle_employee(e: Optional[Employee]) -> None: ...

An optional type is also automatically assumed when the default value is None, for example::

def handle_employee(e: Employee = None): ...

This is equivalent to::

def handle_employee(e: Optional[Employee] = None) -> None: ...

The Any type

A special kind of type is Any. Every class is a subclass of Any. This is also true for the builtin class object. However, to the static type checker these are completely different.

When the type of a value is object, the type checker will reject almost all operations on it, and assigning it to a variable (or using it as a return value) of a more specialized type is a type error. On the other hand, when a value has type Any, the type checker will allow all operations on it, and a value of type Any can be assigned to a variable (or used as a return value) of a more constrained type.

Predefined constants

Some predefined Boolean constants are defined in the typing module to enable platform-specific type definitions and such::

from typing import PY2, PY3, WINDOWS, POSIX

if PY2: text = unicode else: text = str

def f() -> text: ...

if WINDOWS: loop = ProactorEventLoop else: loop = UnixSelectorEventLoop

It is up to the type checker implementation to define their values, as long as PY2 == not PY3 and WINDOWS == not POSIX. When the program is being executed these always reflect the current platform, and this is also the suggested default when the program is being type-checked.

Compatibility with other uses of function annotations

A number of existing or potential use cases for function annotations exist, which are incompatible with type hinting. These may confuse a static type checker. However, since type hinting annotations have no runtime behavior (other than evaluation of the annotation expression and storing annotations in the __annotations__ attribute of the function object), this does not make the program incorrect -- it just may cause a type checker to emit spurious warnings or errors.

To mark portions of the program that should not be covered by type hinting, you can use one or more of the following:

For more details see later sections.

In order for maximal compatibility with offline type checking it may eventually be a good idea to change interfaces that rely on annotations to switch to a different mechanism, for example a decorator. In Python 3.5 there is no pressure to do this, however. See also the longer discussion under Rejected alternatives_ below.

Type comments

No first-class syntax support for explicitly marking variables as being of a specific type is added by this PEP. To help with type inference in complex cases, a comment of the following format may be used::

x = [] # type: List[Employee] x, y, z = [], [], [] # type: List[int], List[int], List[str] x, y, z = [], [], [] # type: (List[int], List[int], List[str]) x = [ 1, 2, ] # type: List[int]

Type comments should be put on the last line of the statement that contains the variable definition. They can also be placed on with statements and for statements, right after the colon.

Examples of type comments on with and for statements::

with frobnicate() as foo: # type: int # Here foo is an int ...

for x, y in points: # type: float, float # Here x and y are floats ...

The # type: ignore comment should be put on the line that the error refers to::

import http.client errors = { 'not_found': http.client.NOT_FOUND # type: ignore }

A # type: ignore comment on a line by itself disables all type checking for the rest of the file.

If type hinting proves useful in general, a syntax for typing variables may be provided in a future Python version.

Casts

Occasionally the type checker may need a different kind of hint: the programmer may know that an expression is of a more constrained type than the type checker infers. For example::

from typing import List, cast

def find_first_str(a: List[object]) -> str: index = next(i for i, x in enumerate(a) if isinstance(x, str)) # We only get here if there's at least one string in a return cast(str, a[index])

The type checker infers the type object for a[index], but we know that (if the code gets to that point) it must be a string. The cast(t, x) call tells the type checker that we are confident that the type of x is t. At runtime a cast always returns the expression unchanged -- it does not check the type, and it does not convert or coerce the value.

Casts differ from type comments (see the previous section). When using a type comment, the type checker should still verify that the inferred type is consistent with the stated type. When using a cast, the type checker should blindly believe the programmer. Also, casts can be used in expressions, while type comments only apply to assignments.

Stub Files

Stub files are files containing type hints that are only for use by the type checker, not at runtime. There are several use cases for stub files:

Stub files have the same syntax as regular Python modules. There is one feature of the typing module that may only be used in stub files: the @overload decorator described below.

The type checker should only check function signatures in stub files; function bodies in stub files should just be a single pass statement.

The type checker should have a configurable search path for stub files. If a stub file is found the type checker should not read the corresponding "real" module.

While stub files are syntactically valid Python modules, they use the .pyi extension to make it possible to maintain stub files in the same directory as the corresponding real module. This also reinforces the notion that no runtime behavior should be expected of stub files.

Function overloading

The @overload decorator allows describing functions that support multiple different combinations of argument types. This pattern is used frequently in builtin modules and types. For example, the __getitem__() method of the bytes type can be described as follows::

from typing import overload

class bytes: ... @overload def getitem(self, i: int) -> int: pass @overload def getitem(self, s: slice) -> bytes: pass

This description is more precise than would be possible using unions (which cannot express the relationship between the argument and return types)::

from typing import Union class bytes: ... def getitem(self, a: Union[int, slice]) -> Union[int, bytes]: pass

Another example where @overload comes in handy is the type of the builtin map() function, which takes a different number of arguments depending on the type of the callable::

from typing import Callable, Iterable, Iterator, Tuple, TypeVar, overload

T1 = TypeVar('T1') T2 = TypeVar('T2) S = TypeVar('S')

@overload def map(func: Callable[[T1], S], iter1: Iterable[T1]) -> Iterator[S]: pass @overload def map(func: Callable[[T1, T2], S], iter1: Iterable[T1], iter2: Iterable[T2]) -> Iterator[S]: pass

... and we could add more items to support more than two iterables

Note that we could also easily add items to support map(None, ...)::

@overload def map(func: None, iter1: Iterable[T1]) -> Iterable[T1]: pass @overload def map(func: None, iter1: Iterable[T1], iter2: Iterable[T2]) -> Iterable[Tuple[T1, T2]]: pass

The @overload decorator may only be used in stub files. While it would be possible to provide a multiple dispatch implementation using this syntax, its implementation would require using sys._getframe(), which is frowned upon. Also, designing and implementing an efficient multiple dispatch mechanism is hard, which is why previous attempts were abandoned in favor of functools.singledispatch(). (See PEP 443, especially its section "Alternative approaches".) In the future we may come up with a satisfactory multiple dispatch design, but we don't want such a design to be constrained by the overloading syntax defined for type hints in stub files.

Storing and distributing stub files

The easiest form of stub file storage and distribution is to put them alongside Python modules in the same directory. This makes them easy to find by both programmers and the tools. However, since package maintainers are free not to add type hinting to their packages, third-party stubs installable by pip from PyPI are also supported. In this case we have to consider three issues: naming, versioning, installation path.

This PEP does not provide a recommendation on a naming scheme that should be used for third-party stub file packages. Discoverability will hopefully be based on package popularity, like with Django packages for example.

Third-party stubs have to be versioned using the lowest version of the source package that is compatible. Example: FooPackage has versions 1.0, 1.1, 1.2, 1.3, 2.0, 2.1, 2.2. There are API changes in versions 1.1, 2.0 and 2.2. The stub file package maintainer is free to release stubs for all versions but at least 1.0, 1.1, 2.0 and 2.2 are needed to enable the end user type check all versions. This is because the user knows that the closest lower or equal version of stubs is compatible. In the provided example, for FooPackage 1.3 the user would choose stubs version 1.1.

Note that if the user decides to use the "latest" available source package, using the "latest" stub files should generally also work if they're updated often.

Third-party stub packages can use any location for stub storage. The type checker will search for them using PYTHONPATH. A default fallback directory that is always checked is shared/typehints/python3.5/ (or 3.6, etc.). Since there can only be one package installed for a given Python version per environment, no additional versioning is performed under that directory (just like bare directory installs by pip in site-packages). Stub file package authors might use the following snippet in setup.py::

... data_files=[ ( 'shared/typehints/python{}.{}'.format(sys.version_info[:2]), pathlib.Path(SRC_PATH).glob('**/.pyi'), ), ], ...

Exceptions

No syntax for listing explicitly raised exceptions is proposed. Currently the only known use case for this feature is documentational, in which case the recommendation is to put this information in a docstring.

The typing Module

To open the usage of static type checking to Python 3.5 as well as older versions, a uniform namespace is required. For this purpose, a new module in the standard library is introduced called typing. It holds a set of classes representing builtin types with generics, namely:

The generic versions of concrete collection types (Dict, List, Set, FrozenSet, and homogeneous arbitrary-length Tuple) are mainly useful for annotating return values. For arguments, prefer the abstract collection types defined below, e.g. Mapping, Sequence or AbstractSet.

The typing module defines the Generator type for return values of generator functions. It is a subtype of Iterable and it has additional type variables for the type accepted by the send() method and the return type of the generator:

It also introduces factories and helper members needed to express generics and union types:

All abstract base classes available in collections.abc are importable from the typing module, with added generics support:

A few one-off types are defined that test for single special methods (similar to Hashable or Sized):

The library includes literals for platform-specific type hinting:

The following conveniece functions and decorators are exported:

The following types are available in the typing.io module:

The following types are provided by the typing.re module:

As a convenience measure, types from typing.io and typing.re are also available in typing (quoting Guido, "There's a reason those modules have two-letter names.").

Rejected Alternatives

During discussion of earlier drafts of this PEP, various objections were raised and alternatives were proposed. We discuss some of these here and explain why we reject them.

Several main objections were raised.

Which brackets for generic type parameters?

Most people are familiar with the use of angular brackets (e.g. List<int>) in languages like C++, Java, C# and Swift to express the parametrization of generic types. The problem with these is that they are really hard to parse, especially for a simple-minded parser like Python. In most languages the ambiguities are usually dealy with by only allowing angular brackets in specific syntactic positions, where general expressions aren't allowed. (And also by using very powerful parsing techniques that can backtrack over an arbitrary section of code.)

But in Python, we'd like type expressions to be (syntactically) the same as other expressions, so that we can use e.g. variable assignment to create type aliases. Consider this simple type expression::

List<int>

From the Python parser's perspective, the expression begins with the same four tokens (NAME, LESS, NAME, GREATER) as a chained comparison::

a < b > c  # I.e., (a < b) and (b > c)

We can even make up an example that could be parsed both ways::

a < b > [ c ]

Assuming we had angular brackets in the language, this could be interpreted as either of the following two::

(a<b>)[c]      # I.e., (a<b>).__getitem__(c)
a < b > ([c])  # I.e., (a < b) and (b > [c])

It would surely be possible to come up with a rule to disambiguate such cases, but to most users the rules would feel arbitrary and complex. It would also require us to dramatically change the CPython parser (and every other parser for Python). It should be noted that Python's current parser is intentionally "dumb" -- a simple grammar is easier for users to reason about.

For all these reasons, square brackets (e.g. List[int]) are (and have long been) the preferred syntax for generic type parameters. They can be implemented by defining the __getitem__() method on the metaclass, and no new syntax is required at all. This option works in all recent versions of Python (starting with Python 2.2). Python is not alone in this syntactic choice -- generic classes in Scala also use square brackets.

What about existing uses of annotations?

One line of argument points out that PEP 3107 explicitly supports the use of arbitrary expressions in function annotations. The new proposal is then considered incompatible with the specification of PEP 3107.

Our response to this is that, first of all, the current proposal does not introduce any direct incompatibilities, so programs using annotations in Python 3.4 will still work correctly and without prejudice in Python 3.5.

We do hope that type hints will eventually become the sole use for annotations, but this will require additional discussion and a deprecation period after the initial roll-out of the typing module with Python 3.5. The current PEP will have provisional status (see PEP 411) until Python 3.6 is released. The fastest conceivable scheme would introduce silent deprecation of non-type-hint annotations in 3.6, full deprecation in 3.7, and declare type hints as the only allowed use of annotations in Python 3.8. This should give authors of packages that use annotations plenty of time to devise another approach, even if type hints become an overnight success.

Another possible outcome would be that type hints will eventually become the default meaning for annotations, but that there will always remain an option to disable them. For this purpose the current proposal defines a decorator @no_type_check which disables the default interpretation of annotations as type hints in a given class or function. It also defines a meta-decorator @no_type_check_decorator which can be used to decorate a decorator (!), causing annotations in any function or class decorated with the latter to be ignored by the type checker.

There are also # type: ignore comments, and static checkers should support configuration options to disable type checking in selected packages.

Despite all these options, proposals have been circulated to allow type hints and other forms of annotations to coexist for individual arguments. One proposal suggests that if an annotation for a given argument is a dictionary literal, each key represents a different form of annotation, and the key 'type' would be use for type hints. The problem with this idea and its variants is that the notation becomes very "noisy" and hard to read. Also, in most cases where existing libraries use annotations, there would be little need to combine them with type hints. So the simpler approach of selectively disabling type hints appears sufficient.

The problem of forward declarations

The current proposal is admittedly sub-optimal when type hints must contain forward references. Python requires all names to be defined by the time they are used. Apart from circular imports this is rarely a problem: "use" here means "look up at runtime", and with most "forward" references there is no problem in ensuring that a name is defined before the function using it is called.

The problem with type hints is that annotations (per PEP 3107, and similar to default values) are evaluated at the time a function is defined, and thus any names used in an annotation must be already defined when the function is being defined. A common scenario is a class definition whose methods need to reference the class itself in their annotations. (More general, it can also occur with mutually recursive classes.) This is natural for container types, for example::

class Node: """Binary tree node."""

  def __init__(self, left: Node, right: None):
      self.left = left
      self.right = right

As written this will not work, because of the peculiarity in Python that class names become defined once the entire body of the class has been executed. Our solution, which isn't particularly elegant, but gets the job done, is to allow using string literals in annotations. Most of the time you won't have to use this though -- most uses of type hints are expected to reference builtin types or types defined in other modules.

A counterproposal would change the semantics of type hints so they aren't evaluated at runtime at all (after all, type checking happens off-line, so why would type hints need to be evaluated at runtime at all). This of course would run afoul of backwards compatibility, since the Python interpreter doesn't actually know whether a particular annotation is meant to be a type hint or something else.

A compromise is possible where a __future__ import could enable turning all annotations in a given module into string literals, as follows::

from future import annotations

class ImSet: def add(self, a: ImSet) -> List[ImSet]: ...

assert ImSet.add.annotations == {'a': 'ImSet', 'return': 'List[ImSet]'}

Such a __future__ import statement will be proposed in a separate PEP.

The double colon

A few creative souls have tried to invent solutions for this problem. For example, it was proposed to use a double colon (::) for type hints, solving two problems at once: disambiguating between type hints and other annotations, and changing the semantics to preclude runtime evaluation. There are several things wrong with this idea, however.

Other forms of new syntax

A few other forms of alternative syntax have been proposed, e.g. the introduction of a where keyword [roberge]_, and Cobra-inspired requires clauses. But these all share a problem with the double colon: they won't work for earlier versions of Python 3. The same would apply to a new __future__ import.

Other backwards compatible conventions

The ideas put forward include:

It's also been proposed to simply wait another release. But what problem would that solve? It would just be procrastination.

PEP Development Process

A live draft for this PEP lives on GitHub [github]. There is also an issue tracker [issues], where much of the technical discussion takes place.

The draft on GitHub is updated regularly in small increments. The official PEPS repo [peps_] is (usually) only updated when a new draft is posted to python-dev.

Acknowledgements

This document could not be completed without valuable input, encouragement and advice from Jim Baker, Jeremy Siek, Michael Matson Vitousek, Andrey Vlasovskikh, Radomir Dopieralski, Peter Ludemann, and the BDFL-Delegate, Mark Shannon.

Influences include existing languages, libraries and frameworks mentioned in PEP 482. Many thanks to their creators, in alphabetical order: Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings, Anders Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer, Raoul-Gabriel Urma, and Julien Verlaguet.

References

.. [mypy] http://mypy-lang.org

.. [pyflakes] https://github.com/pyflakes/pyflakes/

.. [pylint] http://www.pylint.org

.. [gvr-artima] http://www.artima.com/weblogs/viewpost.jsp?thread=85551

.. [roberge] http://aroberge.blogspot.com/2015/01/type-hinting-in-python-focus-on.html

.. [github] https://github.com/ambv/typehinting

.. [issues] https://github.com/ambv/typehinting/issues

.. [peps] https://hg.python.org/peps/file/tip/pep-0484.txt

Copyright

This document has been placed in the public domain.

.. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

-- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20150417/00bbb75a/attachment-0001.html>

More information about the Python-Dev mailing list