functools.rst (original) (raw)

line wrap: on

line source

:mod:functools --- Higher-order functions and operations on callable objects ============================================================================== .. module:: functools :synopsis: Higher-order functions and operations on callable objects. .. moduleauthor:: Peter Harris scav@blueyonder.co.uk .. moduleauthor:: Raymond Hettinger python@rcn.com .. moduleauthor:: Nick Coghlan ncoghlan@gmail.com .. moduleauthor:: Łukasz Langa lukasz@langa.pl .. sectionauthor:: Peter Harris scav@blueyonder.co.uk Source code: :source:Lib/functools.py -------------- The :mod:functools module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module. The :mod:functools module defines the following functions: .. function:: cmp_to_key(func) Transform an old-style comparison function to a key function. Used with tools that accept key functions (such as :func:sorted, :func:min, :func:max, :func:heapq.nlargest, :func:heapq.nsmallest, :func:itertools.groupby). This function is primarily used as a transition tool for programs being converted from Python 2 which supported the use of comparison functions. A comparison function is any callable that accept two arguments, compares them, and returns a negative number for less-than, zero for equality, or a positive number for greater-than. A key function is a callable that accepts one argument and returns another value indicating the position in the desired collation sequence. Example:: sorted(iterable, key=cmp_to_key(locale.strcoll)) # locale-aware sort order .. versionadded:: 3.2 .. decorator:: lru_cache(maxsize=128, typed=False) Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments. Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable. If maxsize is set to None, the LRU feature is disabled and the cache can grow without bound. The LRU feature performs best when maxsize is a power-of-two. If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results. To help measure the effectiveness of the cache and tune the maxsize parameter, the wrapped function is instrumented with a :func:cache_info function that returns a :term:named tuple showing hits, misses, maxsize and currsize. In a multi-threaded environment, the hits and misses are approximate. The decorator also provides a :func:cache_clear function for clearing or invalidating the cache. The original underlying function is accessible through the :attr:__wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache. An LRU (least recently used) cache[](#l74) <http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used>_ works best when the most recent calls are the best predictors of upcoming calls (for example, the most popular articles on a news server tend to change each day). The cache's size limit assures that the cache does not grow without bound on long-running processes such as web servers. Example of an LRU cache for static web content:: @lru_cache(maxsize=32) def get_pep(num): 'Retrieve text of a Python Enhancement Proposal' resource = 'http://www.python.org/dev/peps/pep-%04d/' % num try: with urllib.request.urlopen(resource) as s: return s.read() except urllib.error.HTTPError: return 'Not Found' >>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991: ... pep = get_pep(n) ... print(n, len(pep)) >>> get_pep.cache_info() CacheInfo(hits=3, misses=8, maxsize=32, currsize=8) Example of efficiently computing Fibonacci numbers <http://en.wikipedia.org/wiki/Fibonacci_number> using a cache to implement a dynamic programming <http://en.wikipedia.org/wiki/Dynamic_programming> technique:: @lru_cache(maxsize=None) def fib(n): if n < 2: return n return fib(n-1) + fib(n-2) >>> [fib(n) for n in range(16)] [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610] >>> fib.cache_info() CacheInfo(hits=28, misses=16, maxsize=None, currsize=16) .. versionadded:: 3.2 .. versionchanged:: 3.3 Added the typed option. .. decorator:: total_ordering Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations: The class must define one of :meth:__lt__, :meth:__le__, :meth:__gt__, or :meth:__ge__. In addition, the class should supply an :meth:__eq__ method. For example:: @total_ordering class Student: def eq(self, other): return ((self.lastname.lower(), self.firstname.lower()) == (other.lastname.lower(), other.firstname.lower())) def lt(self, other): return ((self.lastname.lower(), self.firstname.lower()) < (other.lastname.lower(), other.firstname.lower())) .. versionadded:: 3.2 .. function:: partial(func, *args, **keywords) Return a new :class:partial object which when called will behave like *func* called with the positional arguments *args* and keyword arguments *keywords*. If more arguments are supplied to the call, they are appended to *args*. If additional keyword arguments are supplied, they extend and override *keywords*. Roughly equivalent to:: def partial(func, *args, **keywords): def newfunc(*fargs, **fkeywords): newkeywords = keywords.copy() newkeywords.update(fkeywords) return func(*(args + fargs), **newkeywords) newfunc.func = func newfunc.args = args newfunc.keywords = keywords return newfunc The :func:partial is used for partial function application which "freezes" some portion of a function's arguments and/or keywords resulting in a new object with a simplified signature. For example, :func:partial can be used to create a callable that behaves like the :func:int function where the *base* argument defaults to two: >>> from functools import partial >>> basetwo = partial(int, base=2) >>> basetwo.doc = 'Convert base 2 string to an int.' >>> basetwo('10010') 18 .. function:: reduce(function, iterable[, initializer]) Apply function of two arguments cumulatively to the items of sequence, from left to right, so as to reduce the sequence to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the sequence. If the optional initializer is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty. If initializer is not given and sequence contains only one item, the first item is returned. .. decorator:: singledispatch(default) Transforms a function into a single-dispatch generic function. A generic function is composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as single dispatch. To define a generic function, decorate it with the @singledispatch decorator. Note that the dispatch happens on the type of the first argument, create your function accordingly:: >>> from functools import singledispatch >>> @singledispatch ... def fun(arg, verbose=False): ... if verbose: ... print("Let me just say,", end=" ") ... print(arg) To add overloaded implementations to the function, use the :func:register attribute of the generic function. It is a decorator, taking a type parameter and decorating a function implementing the operation for that type:: >>> @fun.register(int) ... def _(arg, verbose=False): ... if verbose: ... print("Strength in numbers, eh?", end=" ") ... print(arg) ... >>> @fun.register(list) ... def _(arg, verbose=False): ... if verbose: ... print("Enumerate this:") ... for i, elem in enumerate(arg): ... print(i, elem) To enable registering lambdas and pre-existing functions, the :func:register attribute can be used in a functional form:: >>> def nothing(arg, verbose=False): ... print("Nothing.") ... >>> fun.register(type(None), nothing) The :func:register attribute returns the undecorated function which enables decorator stacking, pickling, as well as creating unit tests for each variant independently:: >>> @fun.register(float) ... @fun.register(Decimal) ... def fun_num(arg, verbose=False): ... if verbose: ... print("Half of your number:", end=" ") ... print(arg / 2) ... >>> fun_num is fun False When called, the generic function dispatches on the type of the first argument:: >>> fun("Hello, world.") Hello, world. >>> fun("test.", verbose=True) Let me just say, test. >>> fun(42, verbose=True) Strength in numbers, eh? 42 >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True) Enumerate this: 0 spam 1 spam 2 eggs 3 spam >>> fun(None) Nothing. >>> fun(1.23) 0.615 Where there is no registered implementation for a specific type, its method resolution order is used to find a more generic implementation. To check which implementation will the generic function choose for a given type, use the dispatch() attribute:: >>> fun.dispatch(float) <function fun_num at 0x104319058> >>> fun.dispatch(dict) <function fun at 0x103fe4788> To access all registered implementations, use the read-only registry attribute:: >>> fun.registry.keys() dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>, <class 'decimal.Decimal'>, <class 'list'>, <class 'float'>]) >>> fun.registry[float] <function fun_num at 0x1035a2840> >>> fun.registry[object] <function fun at 0x103170788> .. versionadded:: 3.4 .. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES) Update a wrapper function to look like the wrapped function. The optional arguments are tuples to specify which attributes of the original function are assigned directly to the matching attributes on the wrapper function and which attributes of the wrapper function are updated with the corresponding attributes from the original function. The default values for these arguments are the module level constants WRAPPER_ASSIGNMENTS (which assigns to the wrapper function's name, module, annotations and doc, the documentation string) and WRAPPER_UPDATES (which updates the wrapper function's dict, i.e. the instance dictionary). To allow access to the original function for introspection and other purposes (e.g. bypassing a caching decorator such as :func:lru_cache), this function automatically adds a wrapped attribute to the wrapper that refers to the original function. The main intended use for this function is in :term:decorator functions which wrap the decorated function and return the wrapper. If the wrapper function is not updated, the metadata of the returned function will reflect the wrapper definition rather than the original function definition, which is typically less than helpful. :func:update_wrapper may be used with callables other than functions. Any attributes named in assigned or updated that are missing from the object being wrapped are ignored (i.e. this function will not attempt to set them on the wrapper function). :exc:AttributeError is still raised if the wrapper function itself is missing any attributes named in updated. .. versionadded:: 3.2 Automatic addition of the __wrapped__ attribute. .. versionadded:: 3.2 Copying of the __annotations__ attribute by default. .. versionchanged:: 3.2 Missing attributes no longer trigger an :exc:AttributeError. .. decorator:: wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES) This is a convenience function for invoking partial(update_wrapper,[](#l336) wrapped=wrapped, assigned=assigned, updated=updated) as a function decorator when defining a wrapper function. For example: >>> from functools import wraps >>> def my_decorator(f): ... @wraps(f) ... def wrapper(*args, **kwds): ... print('Calling decorated function') ... return f(*args, **kwds) ... return wrapper ... >>> @my_decorator ... def example(): ... """Docstring""" ... print('Called example function') ... >>> example() Calling decorated function Called example function >>> example.name 'example' >>> example.doc 'Docstring' Without the use of this decorator factory, the name of the example function would have been 'wrapper', and the docstring of the original :func:example would have been lost. .. _partial-objects: :class:partial Objects ------------------------ :class:partial objects are callable objects created by :func:partial. They have three read-only attributes: .. attribute:: partial.func A callable object or function. Calls to the :class:partial object will be forwarded to :attr:func with new arguments and keywords. .. attribute:: partial.args The leftmost positional arguments that will be prepended to the positional arguments provided to a :class:partial object call. .. attribute:: partial.keywords The keyword arguments that will be supplied when the :class:partial object is called. :class:partial objects are like :class:function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the :attr:__name__ and :attr:__doc__ attributes are not created automatically. Also, :class:partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.