cpython: 78c8f450b84c (original) (raw)

Mercurial > cpython

changeset 105647:78c8f450b84c 3.6

Issue #28091: Document PEP 525 & PEP 530. Patch by Eric Appelt. [#28091]

Yury Selivanov yury@magic.io
date	Thu, 15 Dec 2016 17:36:05 -0500
parents	e5360d413ce4
children	6bf84e661e69 418ba3a0f090
files	Doc/glossary.rst Doc/library/asyncio-eventloop.rst Doc/library/inspect.rst Doc/library/sys.rst Doc/library/types.rst Doc/reference/compound_stmts.rst Doc/reference/datamodel.rst Doc/reference/expressions.rst Doc/reference/simple_stmts.rst
diffstat	9 files changed, 298 insertions(+), 4 deletions(-)[+] [-] Doc/glossary.rst 28 Doc/library/asyncio-eventloop.rst 18 Doc/library/inspect.rst 21 Doc/library/sys.rst 36 Doc/library/types.rst 8 Doc/reference/compound_stmts.rst 2 Doc/reference/datamodel.rst 19 Doc/reference/expressions.rst 166 Doc/reference/simple_stmts.rst 4

line wrap: on

line diff

--- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -74,6 +74,34 @@ Glossary :keyword:async with statement by defining :meth:__aenter__ and :meth:__aexit__ methods. Introduced by :pep:492.

asynchronous generator

 A function which returns an :term:`asynchronous generator iterator`.  It[](#l1.8)

 looks like a coroutine function defined with :keyword:`async def` except[](#l1.9)

 that it contains :keyword:`yield` expressions for producing a series of[](#l1.10)

 values usable in an :keyword:`async for` loop.[](#l1.11)

 Usually refers to a asynchronous generator function, but may refer to an[](#l1.13)

 *asynchronous generator iterator* in some contexts.  In cases where the[](#l1.14)

 intended meaning isn't clear, using the full terms avoids ambiguity.[](#l1.15)

 An asynchronous generator function may contain :keyword:`await`[](#l1.17)

 expressions as well as :keyword:`async for`, and :keyword:`async with`[](#l1.18)

```
 statements.[](#l1.19)
```

asynchronous generator iterator

 An object created by a :term:`asynchronous generator` function.[](#l1.22)

 This is an :term:`asynchronous iterator` which when called using the[](#l1.24)

 :meth:`__anext__` method returns an awaitable object which will execute[](#l1.25)

 that the body of the asynchronous generator function until the[](#l1.26)

 next :keyword:`yield` expression.[](#l1.27)

 Each :keyword:`yield` temporarily suspends processing, remembering the[](#l1.29)

 location execution state (including local variables and pending[](#l1.30)

 try-statements).  When the *asynchronous generator iterator* effectively[](#l1.31)

 resumes with another awaitable returned by :meth:`__anext__`, it[](#l1.32)

 picks-up where it left-off.  See :pep:`492` and :pep:`525`.[](#l1.33)

+ asynchronous iterable An object, that can be used in an :keyword:async for statement. Must return an :term:asynchronous iterator from its

--- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -88,6 +88,24 @@ Run an event loop This is idempotent and irreversible. No other methods should be called after this one. + +.. coroutinemethod:: AbstractEventLoop.shutdown_asyncgens() +

Schedule all currently open :term:asynchronous generator objects to
close with an :meth:~agen.aclose() call. After calling this method,
the event loop will issue a warning whenever a new asynchronous generator
is iterated. Should be used to finalize all scheduled asynchronous
generators reliably. Example:: +
try:
```
   loop.run_forever()[](#l2.17)
```
finally:

   loop.run_until_complete(loop.shutdown_asyncgens())[](#l2.19)

```
   loop.close()[](#l2.20)
```

.. versionadded:: 3.6 +

+ .. _asyncio-pass-keywords: Calls

--- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -318,6 +318,27 @@ attributes: .. versionadded:: 3.5 +.. function:: isasyncgenfunction(object) +

Return true if the object is an :term:asynchronous generator function,
for example:: +
async def agen():
... yield 1
...
inspect.isasyncgenfunction(agen)
True

.. versionadded:: 3.6 +

+ +.. function:: isasyncgen(object) +

Return true if the object is an :term:asynchronous generator iterator
created by an :term:asynchronous generator function. +
.. versionadded:: 3.6 +

.. function:: istraceback(object) Return true if the object is a traceback.

--- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -594,6 +594,24 @@ always available. .. versionchanged:: 3.6 Added platform_version + +.. function:: get_asyncgen_hooks() +

Returns an asyncgen_hooks object, which is similar to a
:class:~collections.namedtuple of the form (firstiter, finalizer),
where firstiter and finalizer are expected to be either None or
functions which take an :term:asynchronous generator iterator as an
argument, and are used to schedule finalization of an asychronous
generator by an event loop. +
.. versionadded:: 3.6

 See :pep:`525` for more details.[](#l4.18)

.. note::

 This function has been added on a provisional basis (see :pep:`411`[](#l4.21)

```
 for details.)[](#l4.22)
```

+ + .. function:: get_coroutine_wrapper() Returns None, or a wrapper set by :func:set_coroutine_wrapper. @@ -1098,6 +1116,24 @@ always available. implementation platform, rather than part of the language definition, and thus may not be available in all Python implementations. +.. function:: set_asyncgen_hooks(firstiter, finalizer) +

Accepts two optional keyword arguments which are callables that accept an
:term:asynchronous generator iterator as an argument. The firstiter
callable will be called when an asynchronous generator is iterated for the
first time. The finalizer will be called when an asynchronous generator
is about to be garbage collected. +
.. versionadded:: 3.6

 See :pep:`525` for more details, and for a reference example of a[](#l4.41)

 *finalizer* method see the implementation of[](#l4.42)

 ``asyncio.Loop.shutdown_asyncgens`` in[](#l4.43)

 :source:`Lib/asyncio/base_events.py`[](#l4.44)

.. note::

 This function has been added on a provisional basis (see :pep:`411`[](#l4.47)

```
 for details.)[](#l4.48)
```

+ .. function:: set_coroutine_wrapper(wrapper)

--- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -104,6 +104,14 @@ Standard names are defined for the follo .. versionadded:: 3.5 +.. data:: AsyncGeneratorType +

The type of :term:asynchronous generator-iterator objects, created by
asynchronous generator functions. +
.. versionadded:: 3.6 +

+ .. data:: CodeType .. index:: builtin: compile

--- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -697,7 +697,7 @@ coroutine bodies. Functions defined with async def syntax are always coroutine functions, even if they do not contain await or async keywords. -It is a :exc:SyntaxError to use :keyword:yield expressions in +It is a :exc:SyntaxError to use yield from expressions in async def coroutines. An example of a coroutine function::

--- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -627,6 +627,25 @@ Callable types as well as :keyword:async with and :keyword:async for statements. See also the :ref:coroutine-objects section.

Asynchronous generator functions
```
 .. index::[](#l7.8)
```

    single: asynchronous generator; function[](#l7.9)

    single: asynchronous generator; asynchronous iterator[](#l7.10)

 A function or method which is defined using :keyword:`async def` and[](#l7.12)

 which uses the :keyword:`yield` statement is called a[](#l7.13)

 :dfn:`asynchronous generator function`.  Such a function, when called,[](#l7.14)

 returns an asynchronous iterator object which can be used in an[](#l7.15)

 :keyword:`async for` statement to execute the body of the function.[](#l7.16)

 Calling the asynchronous iterator's :meth:`aiterator.__anext__` method[](#l7.18)

 will return an :term:`awaitable` which when awaited[](#l7.19)

 will execute until it provides a value using the :keyword:`yield`[](#l7.20)

 expression.  When the function executes an empty :keyword:`return`[](#l7.21)

 statement or falls off the end, a :exc:`StopAsyncIteration` exception[](#l7.22)

 is raised and the asynchronous iterator will have reached the end of[](#l7.23)

 the set of values to be yielded.[](#l7.24)

+ Built-in functions .. index:: object: built-in function

--- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -172,7 +172,7 @@ Common syntax elements for comprehension .. productionlist:: comprehension: expression comp_for

comp_for: "for" target_list "in" or_test [comp_iter]

comp_for: [ASYNC] "for" target_list "in" or_test [comp_iter] comp_iter: comp_for | comp_if comp_if: "if" expression_nocond [comp_iter]

@@ -186,6 +186,17 @@ each time the innermost block is reached Note that the comprehension is executed in a separate scope, so names assigned to in the target list don't "leak" into the enclosing scope. +Since Python 3.6, in an :keyword:async def function, an :keyword:async for +clause may be used to iterate over a :term:asynchronous iterator. +A comprehension in an :keyword:async def function may consist of either a +:keyword:for or :keyword:async for clause following the leading +expression, may contan additonal :keyword:for or :keyword:async for +clauses, and may also use :keyword:await expressions. +If a comprehension contains either :keyword:async for clauses +or :keyword:await expressions it is called an +:dfn:asynchronous comprehension. An asynchronous comprehension may +suspend the execution of the coroutine function in which it appears. +See also :pep:530. .. _lists: @@ -315,6 +326,14 @@ range(10) for y in bar(x))``. The parentheses can be omitted on calls with only one argument. See section :ref:calls for details. +Since Python 3.6, if the generator appears in an :keyword:async def function, +then :keyword:async for clauses and :keyword:await expressions are permitted +as with an asynchronous comprehension. If a generator expression +contains either :keyword:async for clauses or :keyword:await expressions +it is called an :dfn:asynchronous generator expression. +An asynchronous generator expression yields a new asynchronous +generator object, which is an asynchronous iterator +(see :ref:async-iterators). .. _yieldexpr: @@ -330,9 +349,22 @@ Yield expressions yield_atom: "(" yield_expression ")" yield_expression: "yield" [expression_list | "from" expression] -The yield expression is only used when defining a :term:generator function and +The yield expression is used when defining a :term:generator function +or an :term:asynchronous generator function and thus can only be used in the body of a function definition. Using a yield -expression in a function's body causes that function to be a generator. +expression in a function's body causes that function to be a generator, +and using it in an :keyword:async def function's body causes that +coroutine function to be an asynchronous generator. For example:: +

def gen(): # defines a generator function
```
   yield 123[](#l8.59)
```

async def agen(): # defines an asynchronous generator function (PEP 525)
```
   yield 123[](#l8.62)
```

+ +Generator functions are described below, while asynchronous generator +functions are described separately in section +:ref:asynchronous-generator-functions. When a generator function is called, it returns an iterator known as a generator. That generator then controls the execution of the generator function. @@ -496,6 +528,134 @@ generator functions:: For examples using yield from, see :ref:pep-380 in "What's New in Python." +.. _asynchronous-generator-functions: + +Asynchronous generator functions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The presence of a yield expression in a function or method defined using +:keyword:async def further defines the function as a +:term:asynchronous generator function. + +When an asynchronous generator function is called, it returns an +asynchronous iterator known as an asynchronous generator object. +That object then controls the execution of the generator function. +An asynchronous generator object is typically used in an +:keyword:async for statement in a coroutine function analogously to +how a generator object would be used in a :keyword:for statement. + +Calling one of the asynchronous generator's methods returns an +:term:awaitable object, and the execution starts when this object +is awaited on. At that time, the execution proceeds to the first yield +expression, where it is suspended again, returning the value of +:token:expression_list to the awaiting coroutine. As with a generator, +suspension means that all local state is retained, including the +current bindings of local variables, the instruction pointer, the internal +evaluation stack, and the state of any exception handling. When the execution +is resumed by awaiting on the next object returned by the asynchronous +generator's methods, the function can proceed exactly as if the yield +expression were just another external call. The value of the yield expression +after resuming depends on the method which resumed the execution. If +:meth:~agen.__anext__ is used then the result is :const:None. Otherwise, if +:meth:~agen.asend is used, then the result will be the value passed in to +that method. + +In an asynchronous generator function, yield expressions are allowed anywhere +in a :keyword:try construct. However, if an asynchronous generator is not +resumed before it is finalized (by reaching a zero reference count or by +being garbage collected), then a yield expression within a :keyword:try +construct could result in a failure to execute pending :keyword:finally +clauses. In this case, it is the responsibility of the event loop or +scheduler running the asynchronous generator to call the asynchronous +generator-iterator's :meth:~agen.aclose method and run the resulting +coroutine object, thus allowing any pending :keyword:finally clauses +to execute. + +To take care of finalization, an event loop should define +a finalizer function which takes an asynchronous generator-iterator +and presumably calls :meth:~agen.aclose and executes the coroutine. +This finalizer may be registered by calling :func:sys.set_asyncgen_hooks. +When first iterated over, an asynchronous generator-iterator will store the +registered finalizer to be called upon finalization. For a reference example +of a finalizer method see the implementation of +asyncio.Loop.shutdown_asyncgens in :source:Lib/asyncio/base_events.py. + +The expression yield from <expr> is a syntax error when used in an +asynchronous generator function. + +.. index:: object: asynchronous-generator +.. _asynchronous-generator-methods: + +Asynchronous generator-iterator methods +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This subsection describes the methods of an asynchronous generator iterator, +which are used to control the execution of a generator function. + + +.. index:: exception: StopAsyncIteration + +.. coroutinemethod:: agen.anext() +

Returns an awaitable which when run starts to execute the asynchronous
generator or resumes it at the last executed yield expression. When an
asynchronous generator function is resumed with a :meth:~agen.__anext__
method, the current yield expression always evaluates to :const:None in
the returned awaitable, which when run will continue to the next yield
expression. The value of the :token:expression_list of the yield
expression is the value of the :exc:StopIteration exception raised by
the completing coroutine. If the asynchronous generator exits without
yielding another value, the awaitable instead raises an
:exc:StopAsyncIteration exception, signalling that the asynchronous
iteration has completed. +
This method is normally called implicitly by a :keyword:async for loop. +

+ +.. coroutinemethod:: agen.asend(value) +

Returns an awaitable which when run resumes the execution of the
asynchronous generator. As with the :meth:~generator.send() method for a
generator, this "sends" a value into the asynchronous generator function,
and the value argument becomes the result of the current yield expression.
The awaitable returned by the :meth:asend method will return the next
value yielded by the generator as the value of the raised
:exc:StopIteration, or raises :exc:StopAsyncIteration if the
asynchronous generator exits without yielding another value. When
:meth:asend is called to start the asynchronous
generator, it must be called with :const:None as the argument,
because there is no yield expression that could receive the value. +

+ +.. coroutinemethod:: agen.athrow(type[, value[, traceback]]) +

Returns an awaitable that raises an exception of type type at the point
where the asynchronous generator was paused, and returns the next value
yielded by the generator function as the value of the raised
:exc:StopIteration exception. If the asynchronous generator exits
without yielding another value, an :exc:StopAsyncIteration exception is
raised by the awaitable.
If the generator function does not catch the passed-in exception, or
raises a different exception, then when the awaitalbe is run that exception
propagates to the caller of the awaitable. +

+.. index:: exception: GeneratorExit + + +.. coroutinemethod:: agen.aclose() +

Returns an awaitable that when run will throw a :exc:GeneratorExit into
the asynchronous generator function at the point where it was paused.
If the asynchronous generator function then exits gracefully, is already
closed, or raises :exc:GeneratorExit (by not catching the exception),
then the returned awaitable will raise a :exc:StopIteration exception.
Any further awaitables returned by subsequent calls to the asynchronous
generator will raise a :exc:StopAsyncIteration exception. If the
asynchronous generator yields a value, a :exc:RuntimeError is raised
by the awaitable. If the asynchronous generator raises any other exception,
it is propagated to the caller of the awaitable. If the asynchronous
generator has already exited due to an exception or normal exit, then
further calls to :meth:aclose will return an awaitable that does nothing.

.. _primaries:

--- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -492,6 +492,10 @@ generator is done and will cause :exc:S[](#l9.3) value (if any) is used as an argument to construct :exc:StopIteration and[](#l9.4) becomes the :attr:StopIteration.value attribute.[](#l9.5) [](#l9.6) +In an asynchronous generator function, an empty :keyword:return statement[](#l9.7) +indicates that the asynchronous generator is done and will cause[](#l9.8) +:exc:StopAsyncIteration to be raised. A non-empty :keyword:return` +statement is a syntax error in an asynchronous generator function. .. _yield: