PEP 526 – Syntax for Variable Annotations | peps.python.org (original) (raw)

Author:

Ryan Gonzalez , Philip House , Ivan Levkivskyi , Lisa Roach , Guido van Rossum

Status:

Final

Type:

Standards Track

Topic:

Typing

Created:

09-Aug-2016

Python-Version:

3.6

Post-History:

30-Aug-2016, 02-Sep-2016

Resolution:

Python-Dev message

Table of Contents

Status

This PEP has been provisionally accepted by the BDFL. See the acceptance message for more color:https://mail.python.org/pipermail/python-dev/2016-September/146282.html

Notice for Reviewers

This PEP was drafted in a separate repo:https://github.com/phouse512/peps/tree/pep-0526.

There was preliminary discussion on python-ideas and athttps://github.com/python/typing/issues/258.

Before you bring up an objection in a public forum please at least read the summary of rejected ideas listed at the end of this PEP.

Abstract

PEP 484 introduced type hints, a.k.a. type annotations. While its main focus was function annotations, it also introduced the notion of type comments to annotate variables:

'primes' is a list of integers

primes = [] # type: List[int]

'captain' is a string (Note: initial value is a problem)

captain = ... # type: str

class Starship: # 'stats' is a class variable stats = {} # type: Dict[str, int]

This PEP aims at adding syntax to Python for annotating the types of variables (including class variables and instance variables), instead of expressing them through comments:

primes: List[int] = []

captain: str # Note: no initial value!

class Starship: stats: ClassVar[Dict[str, int]] = {}

PEP 484 explicitly states that type comments are intended to help with type inference in complex cases, and this PEP does not change this intention. However, since in practice type comments have also been adopted for class variables and instance variables, this PEP also discusses the use of type annotations for those variables.

Rationale

Although type comments work well enough, the fact that they’re expressed through comments has some downsides:

The majority of these issues can be alleviated by making the syntax a core part of the language. Moreover, having a dedicated annotation syntax for class and instance variables (in addition to method annotations) will pave the way to static duck-typing as a complement to nominal typing defined by PEP 484.

Non-goals

While the proposal is accompanied by an extension of the typing.get_type_hintsstandard library function for runtime retrieval of annotations, variable annotations are not designed for runtime type checking. Third party packages will have to be developed to implement such functionality.

It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention. Type annotations should not be confused with variable declarations in statically typed languages. The goal of annotation syntax is to provide an easy way to specify structured type metadata for third party tools.

This PEP does not require type checkers to change their type checking rules. It merely provides a more readable syntax to replace type comments.

Specification

Type annotation can be added to an assignment statement or to a single expression indicating the desired type of the annotation target to a third party type checker:

my_var: int my_var = 5 # Passes type check. other_var: int = 'a' # Flagged as error by type checker, # but OK at runtime.

This syntax does not introduce any new semantics beyond PEP 484, so that the following three statements are equivalent:

var = value # type: annotation var: annotation; var = value var: annotation = value

Below we specify the syntax of type annotations in different contexts and their runtime effects.

We also suggest how type checkers might interpret annotations, but compliance to these suggestions is not mandatory. (This is in line with the attitude towards compliance in PEP 484.)

Global and local variable annotations

The types of locals and globals can be annotated as follows:

some_number: int # variable without initial value some_list: List[int] = [] # variable with initial value

Being able to omit the initial value allows for easier typing of variables assigned in conditional branches:

sane_world: bool if 2+2 == 4: sane_world = True else: sane_world = False

Note that, although the syntax does allow tuple packing, it does not allow one to annotate the types of variables when tuple unpacking is used:

Tuple packing with variable annotation syntax

t: Tuple[int, ...] = (1, 2, 3)

or

t: Tuple[int, ...] = 1, 2, 3 # This only works in Python 3.8+

Tuple unpacking with variable annotation syntax

header: str kind: int body: Optional[List[str]] header, kind, body = message

Omitting the initial value leaves the variable uninitialized:

a: int print(a) # raises NameError

However, annotating a local variable will cause the interpreter to always make it a local:

def f(): a: int print(a) # raises UnboundLocalError # Commenting out the a: int makes it a NameError.

as if the code were:

def f(): if False: a = 0 print(a) # raises UnboundLocalError

Duplicate type annotations will be ignored. However, static type checkers may issue a warning for annotations of the same variable by a different type:

a: int a: str # Static type checker may or may not warn about this.

Class and instance variable annotations

Type annotations can also be used to annotate class and instance variables in class bodies and methods. In particular, the value-less notation a: intallows one to annotate instance variables that should be initialized in __init__ or __new__. The proposed syntax is as follows:

class BasicStarship: captain: str = 'Picard' # instance variable with default damage: int # instance variable without default stats: ClassVar[Dict[str, int]] = {} # class variable

Here ClassVar is a special class defined by the typing module that indicates to the static type checker that this variable should not be set on instances.

Note that a ClassVar parameter cannot include any type variables, regardless of the level of nesting: ClassVar[T] and ClassVar[List[Set[T]]] are both invalid if T is a type variable.

This could be illustrated with a more detailed example. In this class:

class Starship: captain = 'Picard' stats = {}

def __init__(self, damage, captain=None):
    self.damage = damage
    if captain:
        self.captain = captain  # Else keep the default

def hit(self):
    Starship.stats['hits'] = Starship.stats.get('hits', 0) + 1

stats is intended to be a class variable (keeping track of many different per-game statistics), while captain is an instance variable with a default value set in the class. This difference might not be seen by a type checker: both get initialized in the class, but captain serves only as a convenient default value for the instance variable, while statsis truly a class variable – it is intended to be shared by all instances.

Since both variables happen to be initialized at the class level, it is useful to distinguish them by marking class variables as annotated with types wrapped in ClassVar[...]. In this way a type checker may flag accidental assignments to attributes with the same name on instances.

For example, annotating the discussed class:

class Starship: captain: str = 'Picard' damage: int stats: ClassVar[Dict[str, int]] = {}

def __init__(self, damage: int, captain: str = None):
    self.damage = damage
    if captain:
        self.captain = captain  # Else keep the default

def hit(self):
    Starship.stats['hits'] = Starship.stats.get('hits', 0) + 1

enterprise_d = Starship(3000) enterprise_d.stats = {} # Flagged as error by a type checker Starship.stats = {} # This is OK

As a matter of convenience (and convention), instance variables can be annotated in __init__ or other methods, rather than in the class:

from typing import Generic, TypeVar T = TypeVar('T')

class Box(Generic[T]): def init(self, content): self.content: T = content

Annotating expressions

The target of the annotation can be any valid single assignment target, at least syntactically (it is up to the type checker what to do with this):

class Cls: pass

c = Cls() c.x: int = 0 # Annotates c.x with int. c.y: int # Annotates c.y with int.

d = {} d['a']: int = 0 # Annotates d['a'] with int. d['b']: int # Annotates d['b'] with int.

Note that even a parenthesized name is considered an expression, not a simple name:

(x): int # Annotates x with int, (x) treated as expression by compiler. (y): int = 0 # Same situation here.

Where annotations aren’t allowed

It is illegal to attempt to annotate variables subject to globalor nonlocal in the same function scope:

def f(): global x: int # SyntaxError

def g(): x: int # Also a SyntaxError global x

The reason is that global and nonlocal don’t own variables; therefore, the type annotations belong in the scope owning the variable.

Only single assignment targets and single right hand side values are allowed. In addition, one cannot annotate variables used in a for or withstatement; they can be annotated ahead of time, in a similar manner to tuple unpacking:

a: int for a in my_iter: ...

f: MyFile with myfunc() as f: ...

Variable annotations in stub files

As variable annotations are more readable than type comments, they are preferred in stub files for all versions of Python, including Python 2.7. Note that stub files are not executed by Python interpreters, and therefore using variable annotations will not lead to errors. Type checkers should support variable annotations in stubs for all versions of Python. For example:

file lib.pyi

ADDRESS: unicode = ...

class Error: cause: Union[str, unicode]

Preferred coding style for variable annotations

Annotations for module level variables, class and instance variables, and local variables should have a single space after corresponding colon. There should be no space before the colon. If an assignment has right hand side, then the equality sign should have exactly one space on both sides. Examples:

Changes to Standard Library and Documentation

Runtime Effects of Type Annotations

Annotating a local variable will cause the interpreter to treat it as a local, even if it was never assigned to. Annotations for local variables will not be evaluated:

def f(): x: NonexistentName # No error.

However, if it is at a module or class level, then the type will be evaluated:

x: NonexistentName # Error! class X: var: NonexistentName # Error!

In addition, at the module or class level, if the item being annotated is a_simple name_, then it and the annotation will be stored in the__annotations__ attribute of that module or class (mangled if private) as an ordered mapping from names to evaluated annotations. Here is an example:

from typing import Dict class Player: ... players: Dict[str, Player] __points: int

print(annotations)

prints: {'players': typing.Dict[str, main.Player],

'_Player__points': <class 'int'>}

__annotations__ is writable, so this is permitted:

annotations['s'] = str

But attempting to update __annotations__ to something other than an ordered mapping may result in a TypeError:

class C: annotations = 42 x: int = 5 # raises TypeError

(Note that the assignment to __annotations__, which is the culprit, is accepted by the Python interpreter without questioning it – but the subsequent type annotation expects it to be aMutableMapping and will fail.)

The recommended way of getting annotations at runtime is by usingtyping.get_type_hints function; as with all dunder attributes, any undocumented use of __annotations__ is subject to breakage without warning:

from typing import Dict, ClassVar, get_type_hints class Starship: hitpoints: int = 50 stats: ClassVar[Dict[str, int]] = {} shield: int = 100 captain: str def init(self, captain: str) -> None: ...

assert get_type_hints(Starship) == {'hitpoints': int, 'stats': ClassVar[Dict[str, int]], 'shield': int, 'captain': str}

assert get_type_hints(Starship.init) == {'captain': str, 'return': None}

Note that if annotations are not found statically, then the__annotations__ dictionary is not created at all. Also the value of having annotations available locally does not offset the cost of having to create and populate the annotations dictionary on every function call. Therefore, annotations at function level are not evaluated and not stored.

Other uses of annotations

While Python with this PEP will not object to:

alice: 'well done' = 'A+' bob: 'what a shame' = 'F-'

since it will not care about the type annotation beyond “it evaluates without raising”, a type checker that encounters it will flag it, unless disabled with # type: ignore or @no_type_check.

However, since Python won’t care what the “type” is, if the above snippet is at the global level or in a class, __annotations__will include {'alice': 'well done', 'bob': 'what a shame'}.

These stored annotations might be used for other purposes, but with this PEP we explicitly recommend type hinting as the preferred use of annotations.

Rejected/Postponed Proposals

Backwards Compatibility

This PEP is fully backwards compatible.

Implementation

An implementation for Python 3.6 can be foundon GitHub.

This document has been placed in the public domain.