[Python-3000] PEP 3107 - Function Annotations (original) (raw)

Tony Lownds tony at PageDNA.com
Thu Dec 28 03:48:03 CET 2006

Previous message: [Python-3000] PEP 3107 - Function Annotations
Next message: [Python-3000] PEP 3107 - Function Annotations
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Dec 27, 2006, at 5:31 PM, Guido van Rossum wrote:

I just noticed that PEP 3107 has quietly been checked in. Thanks Collin and Tony!

After skimming it, I have one observation: the grammar in the PEP doesn't match that implemented by Tony's patch. The difference is only apparent for tuple-unpacking parameters (e.g. b and c in "def foo(a, (b, c), d): pass"). The PEP supports this syntax: def foo((a, b): "something"): ... while the patch supports this instead: def foo((a: "something", b: "somethingelse")): ... (I have to say that I like the patch version better. :-)

Ok, updated (and sent to Neal for checkin, thanks Neal!)

I also note that the PEP uses foo.signature.annotations to access the annotations dict, while the patch uses foo.funcannotations. This is reasonable since we don't have the signature API yet (it's PEP 362, but I don't know its status).

I have changed the language related to PEP 362 so that the signature API isn't restricted unduly.

Finally, the PEP uses Number, Mapping and Sequence as example annotations. I'd rather not use those since they could incorrectly convey the notion that annotations imply type checking semantics, which thety don't (at least not without some kind of decorator).

I've replaced with builtins, but that probably isn't enough of a change. I can work on replacing with arbitrary class names (like A, B, etc).

Here's what I have currently.

Thanks -Tony

PEP: 3107 Title: Function Annotations Version: Revision:53144Revision: 53144 Revision:53144 Last-Modified: Date:2006−12−2207:46:01−0800(Fri,22Dec2006)Date: 2006-12-22 07:46:01 -0800 (Fri, 22 Dec 2006) Date:2006−12−2207:46:01−0800(Fri,22Dec2006) Author: Collin Winter <collinw at gmail.com>, Tony Lownds <tony at lownds.com> Status: Draft Type: Standards Track Requires: 362 Content-Type: text/x-rst Created: 2-Dec-2006 Python-Version: 3.0 Post-History:

Abstract

This PEP introduces a syntax for adding arbitrary metadata annotations to Python functions [#functerm]_.

Rationale

Because Python's 2.x series lacks a standard way of annotating a function's parameters and return values (e.g., with information about a what type a function's return value should be), a variety of tools and libraries have appeared to fill this gap [#tailexamp]_. Some utilise the decorators introduced in "PEP 318", while others parse a function's docstring, looking for annotations there.

This PEP aims to provide a single, standard way of specifying this information, reducing the confusion caused by the wide variation in mechanism and syntax that has existed until this point.

Fundamentals of Function Annotations

Before launching into a discussion of the precise ins and outs of Python 3.0's function annotations, let's first talk broadly about what annotations are and are not:

Function annotations, both for parameters and return values, are completely optional.
Function annotations are nothing more than a way of associating arbitrary Python expressions with various parts of a function at compile-time.

By itself, Python does not attach any particular meaning or significance to annotations. Left to its own, Python simply makes these expressions available as described in Accessing Function Annotations_ below.

The only way that annotations take on meaning is when they are interpreted by third-party libraries. These annotation consumers can do anything they want with a function's annotations. For example, one library might use string-based annotations to provide improved help messages, like so::
```
  def compile(source: "something compilable",
              filename: "where the compilable thing comes from",
              mode: "is this a single statement or a suite?"):
      ...
```
Another library might be used to provide typechecking for Python functions and methods. This library could use annotations to indicate the function's expected input and return types, possibly something like ::
```
  def haul(item: Haulable, *vargs: PackAnimal) -> Distance:
      ...
```
However, neither the strings in the first example nor the type information in the second example have any meaning on their own; meaning comes from third-party libraries alone.
Following from point 2, this PEP makes no attempt to introduce any kind of standard semantics, even for the built-in types. This work will be left to third-party libraries.

There is no worry that these libraries will assign semantics at random, or that a variety of libraries will appear, each with varying semantics and interpretations of what, say, a tuple of strings means. The difficulty inherent in writing annotation interpreting libraries will keep their number low and their authorship in the hands of people who, frankly, know what they're doing.

Syntax

Parameters

Annotations for parameters take the form of optional expressions that follow the parameter name. This example indicates that parameters 'a' and 'c' should both be a int, while parameter 'b' should both be a dict::

 def foo(a: int, b: dict, c: int = 5):
     ...

In pseudo-grammar, parameters now look like identifier [: expression] [= expression]. That is, annotations always precede a parameter's default value and both annotations and default values are optional. Just like how equal signs are used to indicate a default value, colons are used to mark annotations. All annotation expressions are evaluated when the function definition is executed.

Annotations for excess parameters (i.e., *args and **kwargs) are indicated similarly. In the following function definition, *args is flagged as a tuple of int, and **kwargs is marked as a dict whose keys are strings and whose values are of type str. ::

 def foo(*args: int, **kwargs: str):
     ...

Note that, depending on what annotation-interpreting library you're using, the following might also be a valid spelling of the above::

 def foo(*args: [int], **kwargs: {str: str}):
     ...

Only the first, however, has the BDFL's blessing [#blessedexcess]_ as the One Obvious Way.

Return Values

The examples thus far have omitted examples of how to annotate the type of a function's return value. This is done like so::

 def sum(*args: int) -> int:
     ...

The parameter list can now be followed by a literal -> and a Python expression. Like the annotations for parameters, this expression will be evaluated when the function definition is executed.

The grammar for function definitions [#grammar]_ is now::

 decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
 decorators: decorator+
 funcdef: [decorators] 'def' NAME parameters ['->' test] ':' suite
 parameters: '(' [typedargslist] ')'
 typedargslist: ((tfpdef ['=' test] ',')*
                 ('*' [tname] (',' tname ['=' test])* [',' '**'

tname] | '' tname) | tfpdef ['=' test] (',' tfpdef ['=' test]) [',']) tname: NAME [':' test] tfpdef: tname | '(' tfplist ')' tfplist: tfpdef (',' tfpdef) [',']

Lambda

lambda's syntax does not support annotations. The syntax of lambda could be changed to support annotations, by requiring parentheses around the parameter list. However it was decided [#lambda]_ not to make this change because:

It would be an incompatible change.
Lambda's are neutered anyway.
The lambda can always be changed to a function.

Accessing Function Annotations

Once compiled, a function's annotations are available via the function's func_annotations attribute. This attribute is a dictionary, mapping parameter names to an object representing the evaluated annotation expression

There is a special key in the func_annotations mapping, "return". This key is present only if an annotation was supplied for the function's return value.

For example, the following annotation::

 def foo(a: 'x', b: 5 + 6, c: list) -> str:
     ...

would result in a func_annotation mapping of ::

 {'a': 'x',
  'b': 11,
  'c': list,
  'return': str}

The return key was chosen because it cannot conflict with the name of a parameter; any attempt to use return as a parameter name would result in a SyntaxError.

func_annotations is an empty dictionary if no there are no annotations on the function. func_annotations is always an empty dictionary for functions created from lambda expressions.

Standard Library

pydoc and inspect

The pydoc module should display the function annotations when displaying help for a function. The inspect module should change to support annotations.

Relation to Other PEPs

Function Signature Objects [#pep-362]_

Function Signature Objects should expose the function's annotations. The Parameter object may change or other changes may be warranted.

Implementation

A sample implementation for the syntax changes has been provided [#implementation]_ by Tony Lownds.

Rejected Proposals

The BDFL rejected the author's idea for a special syntax for adding annotations to generators as being "too ugly" [#rejectgensyn]_.
Though discussed early on ([#threadgen], [#threadhof]), including special objects in the stdlib for annotating generator functions and higher-order functions was ultimately rejected as being more appropriate for third-party libraries; including them in the standard library raised too many thorny issues.
Despite considerable discussion about a standard type parameterisation syntax, it was decided that this should also be left to third-party libraries. ([#threadimmlist], [#threadmixing], [#emphasistpls]_)