[Python-Dev] pep 362 - 5th edition (original) (raw)

Yury Selivanov yselivanov.ml at gmail.com
Tue Jun 19 17:27:20 CEST 2012

Hello,

The new revision of PEP 362 has been posted: http://www.python.org/dev/peps/pep-0362/

Summary:

  1. What was 'Signature.deepcopy' is now 'Signature.copy'. copy creates a shallow copy of Signature, shallow copying its Parameters as well.

  2. 'Signature.format()' was removed. I think we'll add something to customize formatting later, in 3.4. Although, Signature still has its str method.

  3. Built-in ('C') functions no longer have mutable 'signature' attribute, that patch was reverted. In the "Design Considerations" section we stated clear that we don't support some callables.

  4. Positions of keyword-only parameters now longer affect equality testing of Signatures, i.e. 'foo(, a, b)' is equal to 'foo(, b, a)' (Thanks to Jim Jewett for pointing that out)

The only question we have now is: when we do equality test between Signatures, should we account for positional-only, var_positional and var_keyword arguments names? So that: 'foo(*args)' will be equal to 'bar(*arguments)', but not to 'spam(*coordinates:int)' (Again, I think that's a Jim's idea)

Thank you!

PEP: 362 Title: Function Signature Object Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Brett Cannon <brett at python.org>, Jiwon Seo <seojiwon at gmail.com>, Yury Selivanov <yselivanov at sprymix.com>, Larry Hastings <larry at hastings.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 21-Aug-2006 Python-Version: 3.3 Post-History: 04-Jun-2012

Abstract

Python has always supported powerful introspection capabilities, including introspecting functions and methods (for the rest of this PEP, "function" refers to both functions and methods). By examining a function object you can fully reconstruct the function's signature. Unfortunately this information is stored in an inconvenient manner, and is spread across a half-dozen deeply nested attributes.

This PEP proposes a new representation for function signatures. The new representation contains all necessary information about a function and its parameters, and makes introspection easy and straightforward.

However, this object does not replace the existing function metadata, which is used by Python itself to execute those functions. The new metadata object is intended solely to make function introspection easier for Python programmers.

Signature Object

A Signature object represents the call signature of a function and its return annotation. For each parameter accepted by the function it stores a Parameter object_ in its parameters collection.

A Signature object has the following public attributes and methods:

It's possible to test Signatures for equality. Two signatures are equal when their parameters are equal, their positional and positional-only parameters appear in the same order, and they have equal return annotations.

Changes to the Signature object, or to any of its data members, do not affect the function itself.

Signature also implements __str__ and __copy__ methods. The latter creates a shallow copy of Signature, with all Parameter objects copied as well.

Parameter Object

Python's expressive syntax means functions can accept many different kinds of parameters with many subtle semantic differences. We propose a rich Parameter object designed to represent any possible function parameter.

The structure of the Parameter object is:

Two parameters are equal when they have equal names, kinds, defaults, and annotations.

BoundArguments Object

Result of a Signature.bind call. Holds the mapping of arguments to the function's parameters.

Has the following public attributes:

The arguments attribute should be used in conjunction with Signature.parameters for any arguments processing purposes.

args and kwargs properties can be used to invoke functions: :: def test(a, *, b): ...

sig = signature(test)
ba = sig.bind(10, b=20)
test(*ba.args, **ba.kwargs)

Implementation

The implementation adds a new function signature() to the inspect module. The function is the preferred way of getting a Signature for a callable object.

The function implements the following algorithm:

- If the object is not callable - raise a TypeError

- If the object has a ``__signature__`` attribute and if it
  is not ``None`` - return a shallow copy of it

- If it has a ``__wrapped__`` attribute, return
  ``signature(object.__wrapped__)``

- If the object is a an instance of ``FunctionType`` construct
  and return a new ``Signature`` for it

- If the object is a method or a classmethod, construct and return
  a new ``Signature`` object, with its first parameter (usually
  ``self`` or ``cls``) removed

- If the object is a staticmethod, construct and return
  a new ``Signature`` object

- If the object is an instance of ``functools.partial``, construct
  a new ``Signature`` from its ``partial.func`` attribute, and
  account for already bound ``partial.args`` and ``partial.kwargs``

- If the object is a class or metaclass:

    - If the object's type has a ``__call__`` method defined in
      its MRO, return a Signature for it

    - If the object has a ``__new__`` method defined in its class,
      return a Signature object for it

    - If the object has a ``__init__`` method defined in its class,
      return a Signature object for it

- Return ``signature(object.__call__)``

Note, that the Signature object is created in a lazy manner, and is not automatically cached. If, however, the Signature object was explicitly cached by the user, signature() returns a new shallow copy of it on each invocation.

An implementation for Python 3.3 can be found at [#impl]. The python issue tracking the patch is [#issue].

Design Considerations

No implicit caching of Signature objects

The first PEP design had a provision for implicit caching of Signature objects in the inspect.signature() function. However, this has the following downsides:

Some functions may not be introspectable

Some functions may not be introspectable in certain implementations of Python. For example, in CPython, builtin functions defined in C provide no metadata about their arguments. Adding support for them is out of scope for this PEP.

Examples

Visualizing Callable Objects' Signature

Let's define some classes and functions:

:: from inspect import signature from functools import partial, wraps

class FooMeta(type):
    def __new__(mcls, name, bases, dct, *, bar:bool=False):
        return super().__new__(mcls, name, bases, dct)

    def __init__(cls, name, bases, dct, **kwargs):
        return super().__init__(name, bases, dct)


class Foo(metaclass=FooMeta):
    def __init__(self, spam:int=42):
        self.spam = spam

    def __call__(self, a, b, *, c) -> tuple:
        return a, b, c


def shared_vars(*shared_args):
    """Decorator factory that defines shared variables that are
       passed to every invocation of the function"""

    def decorator(f):
        @wraps(f)
        def wrapper(*args, **kwds):
            full_args = shared_args + args
            return f(*full_args, **kwds)
        # Override signature
        sig = wrapper.__signature__ = signature(f)
        for __ in shared_args:
            sig.parameters.popitem(last=False)
        return wrapper
    return decorator


@shared_vars({})
def example(_state, a, b, c):
    return _state, a, b, c


def format_signature(obj):
    return str(signature(obj))

Now, in the python REPL:

:: >>> format_signature(FooMeta) '(name, bases, dct, *, bar:bool=False)'

>>> format_signature(Foo)
'(spam:int=42)'

>>> format_signature(Foo.__call__)
'(self, a, b, *, c) -> tuple'

>>> format_signature(Foo().__call__)
'(a, b, *, c) -> tuple'

>>> format_signature(partial(Foo().__call__, 1, c=3))
'(b, *, c=3) -> tuple'

>>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20))
'(*, c=20) -> tuple'

>>> format_signature(example)
'(a, b, c)'

>>> format_signature(partial(example, 1, 2))
'(c)'

>>> format_signature(partial(partial(example, 1, b=2), c=3))
'(b=2, c=3)'

Annotation Checker

:: import inspect import functools

def checktypes(func):
    '''Decorator to verify arguments and return types

    Example:

        >>> @checktypes
        ... def test(a:int, b:str) -> int:
        ...     return int(a * b)

        >>> test(10, '1')
        1111111111

        >>> test(10, 1)
        Traceback (most recent call last):
          ...
        ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
    '''

    sig = inspect.signature(func)

    types = {}
    for param in sig.parameters.values():
        # Iterate through function's parameters and build the list of
        # arguments types
        try:
            type_ = param.annotation
        except AttributeError:
            continue
        else:
            if not inspect.isclass(type_):
                # Not a type, skip it
                continue

            types[param.name] = type_

            # If the argument has a type specified, let's check that its
            # default value (if present) conforms with the type.
            try:
                default = param.default
            except AttributeError:
                continue
            else:
                if not isinstance(default, type_):
                    raise ValueError("{func}: wrong type of a default value for {arg!r}". \
                                     format(func=func.__qualname__, arg=param.name))

    def check_type(sig, arg_name, arg_type, arg_value):
        # Internal function that encapsulates arguments type checking
        if not isinstance(arg_value, arg_type):
            raise ValueError("{func}: wrong type of {arg!r} argument, " \
                             "{exp!r} expected, got {got!r}". \
                             format(func=func.__qualname__, arg=arg_name,
                                    exp=arg_type.__name__, got=type(arg_value).__name__))

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        # Let's bind the arguments
        ba = sig.bind(*args, **kwargs)
        for arg_name, arg in ba.arguments.items():
            # And iterate through the bound arguments
            try:
                type_ = types[arg_name]
            except KeyError:
                continue
            else:
                # OK, we have a type for the argument, lets get the corresponding
                # parameter description from the signature object
                param = sig.parameters[arg_name]
                if param.kind == param.VAR_POSITIONAL:
                    # If this parameter is a variable-argument parameter,
                    # then we need to check each of its values
                    for value in arg:
                        check_type(sig, arg_name, type_, value)
                elif param.kind == param.VAR_KEYWORD:
                    # If this parameter is a variable-keyword-argument parameter:
                    for subname, value in arg.items():
                        check_type(sig, arg_name + ':' + subname, type_, value)
                else:
                    # And, finally, if this parameter a regular one:
                    check_type(sig, arg_name, type_, arg)

        result = func(*ba.args, **ba.kwargs)
        # The last bit - let's check that the result is correct
        try:
            return_type = sig.return_annotation
        except AttributeError:
            # Looks like we don't have any restriction on the return type
            pass
        else:
            if isinstance(return_type, type) and not isinstance(result, return_type):
                raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
                                 format(func=func.__qualname__, exp=return_type.__name__,
                                        got=type(result).__name__))
        return result

    return wrapper

References

.. [#impl] pep362 branch (https://bitbucket.org/1st1/cpython/overview) .. [#issue] issue 15008 (http://bugs.python.org/issue15008)

Copyright

This document has been placed in the public domain.

.. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

More information about the Python-Dev mailing list