[Python-Dev] PEP 525 (original) (raw)

Yury Selivanov yselivanov.ml at gmail.com
Tue Aug 23 11:56:25 EDT 2016

Hi,

I think it's time to discuss PEP 525 on python-dev (pasted below).

There were no changes in the PEP since I posted it to python-ideas a couple of weeks ago.

One really critical thing that will block PEP acceptance is asynchronous generators (AG) finalization. The problem is to provide a reliable way to correctly close all AGs on program shutdown.

To recap: PEP 492 requires an event loop or a coroutine runner to run coroutines. PEP 525 defines AGs using asynchronous iteration protocol, also defined in PEP 492. AGs require an 'async for' statement to iterate over them, which in turn can only be used in a coroutine. Therefore, AGs also require an event loop or a coroutine runner to operate.

The finalization problem is related to partial iteration. For instance, let's look at an ordinary synchronous generator:

def gen(): try: while True: yield 42 finally: print("done")

g = gen() next(g) next(g) del g

In the above example, when 'g' is GCed, the interpreter will try to close the generator. It will do that by throwing a GeneratorExit exception into 'g', which would trigger the 'finally' statement.

For AGs we have a similar problem. Except that they can have await expressions in their finally statements, which means that the interpreter can't close them on its own. An event loop is required to run an AG, and an event loop is required to close it correctly.

To enable correct AGs finalization, PEP 525 proposes to add a sys.set_asyncgen_finalizer API. The idea is to have a finalizer callback assigned to each AG, and when it's time to close the AG, that callback will be called. The callback will be installed by the event loop (or coroutine runner), and should schedule a correct asynchronous finalization of the AG (remember, AGs can have 'await' expressions in their finally statements).

The problem with 'set_asyncgen_finalizer' is that the event loop doesn't know about AGs until they are GCed. This can be a problem if we want to write a program that gracefully closes all AGs when the event loop is being closed.

There is an alternative approach to finalization of AGs: instead of assigning a finalizer callback to an AG, we can add an API to intercept AG first iteration. That would allow event loops to have weak references to all AGs running under their control:

  1. that would make it possible to intercept AGs garbage collection similarly to the currently proposed set_asyncgen_finalizer

  2. it would also allow us to implement 'loop.shutdown' coroutine, which would try to asynchronously close all open AGs.

The second approach gives event loops more control and allows to implement APIs to collect open resources gracefully. The only downside is that it's a bit harder for event loops to work with.

Let's discuss.

PEP: 525 Title: Asynchronous Generators Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Yury Selivanov <yury at magic.io> Discussions-To: <python-dev at python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Jul-2016 Python-Version: 3.6 Post-History: 02-Aug-2016

Abstract

PEP 492 introduced support for native coroutines and async/await syntax to Python 3.5. It is proposed here to extend Python's asynchronous capabilities by adding support for asynchronous generators.

Rationale and Goals

Regular generators (introduced in PEP 255) enabled an elegant way of writing complex data producers and have them behave like an iterator.

However, currently there is no equivalent concept for the asynchronous iteration protocol (async for). This makes writing asynchronous data producers unnecessarily complex, as one must define a class that implements __aiter__ and __anext__ to be able to use it in an async for statement.

Essentially, the goals and rationale for PEP 255, applied to the asynchronous execution case, hold true for this proposal as well.

Performance is an additional point for this proposal: in our testing of the reference implementation, asynchronous generators are 2x faster than an equivalent implemented as an asynchronous iterator.

As an illustration of the code quality improvement, consider the following class that prints numbers with a given delay once iterated::

 class Ticker:
     """Yield numbers from 0 to `to` every `delay` seconds."""

     def __init__(self, delay, to):
         self.delay = delay
         self.i = 0
         self.to = to

     def __aiter__(self):
         return self

     async def __anext__(self):
         i = self.i
         if i >= self.to:
             raise StopAsyncIteration
         self.i += 1
         if i:
             await asyncio.sleep(self.delay)
         return i

The same can be implemented as a much simpler asynchronous generator::

 async def ticker(delay, to):
     """Yield numbers from 0 to `to` every `delay` seconds."""
     for i in range(to):
         yield i
         await asyncio.sleep(delay)

Specification

This proposal introduces the concept of asynchronous generators to Python.

This specification presumes knowledge of the implementation of generators and coroutines in Python (PEP 342, PEP 380 and PEP 492).

Asynchronous Generators

A Python generator is any function containing one or more yield expressions::

 def func():            # a function
     return

 def genfunc():         # a generator function
     yield

We propose to use the same approach to define asynchronous generators::

 async def coro():      # a coroutine function
     await smth()

 async def asyncgen():  # an asynchronous generator function
     await smth()
     yield 42

The result of calling an asynchronous generator function is an asynchronous generator object, which implements the asynchronous iteration protocol defined in PEP 492.

It is a SyntaxError to have a non-empty return statement in an asynchronous generator.

Support for Asynchronous Iteration Protocol

The protocol requires two special methods to be implemented:

  1. An __aiter__ method returning an asynchronous iterator.
  2. An __anext__ method returning an awaitable object, which uses StopIteration exception to "yield" values, and StopAsyncIteration exception to signal the end of the iteration.

Asynchronous generators define both of these methods. Let's manually iterate over a simple asynchronous generator::

 async def genfunc():
     yield 1
     yield 2

 gen = genfunc()

 assert gen.__aiter__() is gen

 assert await gen.__anext__() == 1
 assert await gen.__anext__() == 2

 await gen.__anext__()  # This line will raise StopAsyncIteration.

Finalization

PEP 492 requires an event loop or a scheduler to run coroutines. Because asynchronous generators are meant to be used from coroutines, they also require an event loop to run and finalize them.

Asynchronous generators can have try..finally blocks, as well as async with. It is important to provide a guarantee that, even when partially iterated, and then garbage collected, generators can be safely finalized. For example::

 async def square_series(con, to):
     async with con.transaction():
         cursor = con.cursor(
             'SELECT generate_series(0, $1) AS i', to)
         async for row in cursor:
             yield row['i'] ** 2

 async for i in square_series(con, 1000):
     if i == 100:
         break

The above code defines an asynchronous generator that uses async with to iterate over a database cursor in a transaction. The generator is then iterated over with async for, which interrupts the iteration at some point.

The square_series() generator will then be garbage collected, and without a mechanism to asynchronously close the generator, Python interpreter would not be able to do anything.

To solve this problem we propose to do the following:

  1. Implement an aclose method on asynchronous generators returning a special awaitable. When awaited it throws a GeneratorExit into the suspended generator and iterates over it until either a GeneratorExit or a StopAsyncIteration occur.

    This is very similar to what the close() method does to regular Python generators, except that an event loop is required to execute aclose().

  2. Raise a RuntimeError, when an asynchronous generator executes a yield expression in its finally block (using await is fine, though)::

      async def gen():
      try:
          yield
      finally:
          await asyncio.sleep(1)   # Can use 'await'.

          yield                    # Cannot use 'yield',
                                   # this line will trigger a
                                   # RuntimeError.

  3. Add two new methods to the sys module: set_asyncgen_finalizer() and get_asyncgen_finalizer().

The idea behind sys.set_asyncgen_finalizer() is to allow event loops to handle generators finalization, so that the end user does not need to care about the finalization problem, and it just works.

When an asynchronous generator is iterated for the first time, it stores a reference to the current finalizer. If there is none, a RuntimeError is raised. This provides a strong guarantee that every asynchronous generator object will always have a finalizer installed by the correct event loop.

When an asynchronous generator is about to be garbage collected, it calls its cached finalizer. The assumption is that the finalizer will schedule an aclose() call with the loop that was active when the iteration started.

For instance, here is how asyncio is modified to allow safe finalization of asynchronous generators::

# asyncio/base_events.py

class BaseEventLoop:

    def run_forever(self):
        ...
        old_finalizer = sys.get_asyncgen_finalizer()
        sys.set_asyncgen_finalizer(self._finalize_asyncgen)
        try:
            ...
        finally:
            sys.set_asyncgen_finalizer(old_finalizer)
            ...

    def _finalize_asyncgen(self, gen):
        self.create_task(gen.aclose())

sys.set_asyncgen_finalizer() is thread-specific, so several event loops running in parallel threads can use it safely.

Asynchronous Generator Object

The object is modeled after the standard Python generator object. Essentially, the behaviour of asynchronous generators is designed to replicate the behaviour of synchronous generators, with the only difference in that the API is asynchronous.

The following methods and properties are defined:

  1. agen.__aiter__(): Returns agen.

  2. agen.__anext__(): Returns an awaitable, that performs one asynchronous generator iteration when awaited.

  3. agen.asend(val): Returns an awaitable, that pushes the val object in the agen generator. When the agen has not yet been iterated, val must be None.

    Example::

     async def gen():
     await asyncio.sleep(0.1)
     v = yield 42
     print(v)
     await asyncio.sleep(0.2)

 g = gen()

 await g.asend(None)      # Will return 42 after sleeping
                          # for 0.1 seconds.

 await g.asend('hello')   # Will print 'hello' and
                          # raise StopAsyncIteration
                          # (after sleeping for 0.2 seconds.)

  4. agen.athrow(typ, [val, [tb]]): Returns an awaitable, that throws an exception into the agen generator.

    Example::

     async def gen():
     try:
         await asyncio.sleep(0.1)
         yield 'hello'
     except ZeroDivisionError:
         await asyncio.sleep(0.2)
         yield 'world'

 g = gen()
 v = await g.asend(None)
 print(v)                # Will print 'hello' after
                         # sleeping for 0.1 seconds.

 v = await g.athrow(ZeroDivisionError)
 print(v)                # Will print 'world' after
                         $ sleeping 0.2 seconds.

  5. agen.aclose(): Returns an awaitable, that throws a GeneratorExit exception into the generator. The awaitable can either return a yielded value, if agen handled the exception, or agen will be closed and the exception will propagate back to the caller.

  6. agen.__name__ and agen.__qualname__: readable and writable name and qualified name attributes.

  7. agen.ag_await: The object that agen is currently awaiting on, or None. This is similar to the currently available gi_yieldfrom for generators and cr_await for coroutines.

  8. agen.ag_frame, agen.ag_running, and agen.ag_code: defined in the same way as similar attributes of standard generators.

StopIteration and StopAsyncIteration are not propagated out of asynchronous generators, and are replaced with a RuntimeError.

Implementation Details

Asynchronous generator object (PyAsyncGenObject) shares the struct layout with PyGenObject. In addition to that, the reference implementation introduces three new objects:

  1. PyAsyncGenASend: the awaitable object that implements __anext__ and asend() methods.

  2. PyAsyncGenAThrow: the awaitable object that implements athrow() and aclose() methods.

  3. _PyAsyncGenWrappedValue: every directly yielded object from an asynchronous generator is implicitly boxed into this structure. This is how the generator implementation can separate objects that are yielded using regular iteration protocol from objects that are yielded using asynchronous iteration protocol.

PyAsyncGenASend and PyAsyncGenAThrow are awaitables (they have __await__ methods returning self) and are coroutine-like objects (implementing __iter__, __next__, send() and throw() methods). Essentially, they control how asynchronous generators are iterated:

.. image:: pep-0525-1.png :align: center :width: 80%

PyAsyncGenASend and PyAsyncGenAThrow ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

PyAsyncGenASend is a coroutine-like object that drives __anext__ and asend() methods and implements the asynchronous iteration protocol.

agen.asend(val) and agen.__anext__() return instances of PyAsyncGenASend (which hold references back to the parent agen object.)

The data flow is defined as follows:

  1. When PyAsyncGenASend.send(val) is called for the first time, val is pushed to the parent agen object (using existing facilities of PyGenObject.)

    Subsequent iterations over the PyAsyncGenASend objects, push None to agen.

    When a _PyAsyncGenWrappedValue object is yielded, it is unboxed, and a StopIteration exception is raised with the unwrapped value as an argument.

  2. When PyAsyncGenASend.throw(*exc) is called for the first time, *exc is throwed into the parent agen object.

    Subsequent iterations over the PyAsyncGenASend objects, push None to agen.

    When a _PyAsyncGenWrappedValue object is yielded, it is unboxed, and a StopIteration exception is raised with the unwrapped value as an argument.

  3. return statements in asynchronous generators raise StopAsyncIteration exception, which is propagated through PyAsyncGenASend.send() and PyAsyncGenASend.throw() methods.

PyAsyncGenAThrow is very similar to PyAsyncGenASend. The only difference is that PyAsyncGenAThrow.send(), when called first time, throws an exception into the parent agen object (instead of pushing a value into it.)

New Standard Library Functions and Types

  1. types.AsyncGeneratorType -- type of asynchronous generator object.

  2. sys.set_asyncgen_finalizer() and sys.get_asyncgen_finalizer() methods to set up asynchronous generators finalizers in event loops.

  3. inspect.isasyncgen() and inspect.isasyncgenfunction() introspection functions.

Backwards Compatibility

The proposal is fully backwards compatible.

In Python 3.5 it is a SyntaxError to define an async def function with a yield expression inside, therefore it's safe to introduce asynchronous generators in 3.6.

Performance

Regular Generators

There is no performance degradation for regular generators. The following micro benchmark runs at the same speed on CPython with and without asynchronous generators::

 def gen():
     i = 0
     while i < 100000000:
         yield i
         i += 1

 list(gen())

Improvements over asynchronous iterators

The following micro-benchmark shows that asynchronous generators are about 2.3x faster than asynchronous iterators implemented in pure Python::

 N = 10 ** 7

 async def agen():
     for i in range(N):
         yield i

 class AIter:
     def __init__(self):
         self.i = 0

     def __aiter__(self):
         return self

     async def __anext__(self):
         i = self.i
         if i >= N:
             raise StopAsyncIteration
         self.i += 1
         return i

Design Considerations

aiter() and anext() builtins

Originally, PEP 492 defined __aiter__ as a method that should return an awaitable object, resulting in an asynchronous iterator.

However, in CPython 3.5.2, __aiter__ was redefined to return asynchronous iterators directly. To avoid breaking backwards compatibility, it was decided that Python 3.6 will support both ways: __aiter__ can still return an awaitable with a DeprecationWarning being issued.

Because of this dual nature of __aiter__ in Python 3.6, we cannot add a synchronous implementation of aiter() built-in. Therefore, it is proposed to wait until Python 3.7.

Asynchronous list/dict/set comprehensions

Syntax for asynchronous comprehensions is unrelated to the asynchronous generators machinery, and should be considered in a separate PEP.

Asynchronous yield from

While it is theoretically possible to implement yield from support for asynchronous generators, it would require a serious redesign of the generators implementation.

yield from is also less critical for asynchronous generators, since there is no need provide a mechanism of implementing another coroutines protocol on top of coroutines. And to compose asynchronous generators a simple async for loop can be used::

 async def g1():
     yield 1
     yield 2

 async def g2():
     async for v in g1():
         yield v

Why the asend() and athrow() methods are necessary

They make it possible to implement concepts similar to contextlib.contextmanager using asynchronous generators. For instance, with the proposed design, it is possible to implement the following pattern::

 @async_context_manager
 async def ctx():
     await open()
     try:
         yield
     finally:
         await close()

 async with ctx():
     await ...

Another reason is that it is possible to push data and throw exceptions into asynchronous generators using the object returned from __anext__ object, but it is hard to do that correctly. Adding explicit asend() and athrow() will pave a safe way to accomplish that.

In terms of implementation, asend() is a slightly more generic version of __anext__, and athrow() is very similar to aclose(). Therefore having these methods defined for asynchronous generators does not add any extra complexity.

Example

A working example with the current reference implementation (will print numbers from 0 to 9 with one second delay)::

 async def ticker(delay, to):
     for i in range(to):
         yield i
         await asyncio.sleep(delay)


 async def run():
     async for i in ticker(1, 10):
         print(i)


 import asyncio
 loop = asyncio.get_event_loop()
 try:
     loop.run_until_complete(run())
 finally:
     loop.close()

Implementation

The complete reference implementation is available at [1]_.

References

.. [1] https://github.com/1st1/cpython/tree/async_gen

Copyright

This document has been placed in the public domain.

.. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

More information about the Python-Dev mailing list