[Python-Dev] PEP 492: async/await in Python; version 4 (original) (raw)

Guido van Rossum guido at python.org
Tue May 5 23:50:23 CEST 2015

On Tue, May 5, 2015 at 2:29 PM, Paul Moore <p.f.moore at gmail.com> wrote:

On 5 May 2015 at 22:12, Guido van Rossum <guido at python.org> wrote: > I apologize for the confusing documentation. We need more help from > qualified tech writers! Writing PEP 3156 was a huge undertaking for me; > after that I was exhausted and did not want to take on writing the end user > documentation as well, so it was left unfinished. :-(

Fair enough. When I properly document one of my projects, then I'll think about complaining :-) These things happen. > In PEP 3156 (asyncio package) there are really three separate concepts: > > - Future, which is a specific class (of which Task is a subclass); > > - coroutine, by which in this context is meant a generator object obtained > by calling a generator function decorated with @asyncio.coroutine and > written to conform to the asyncio protocol for coroutines (i.e. don't use > bare yield, only use yield from, and the latter always with either a Future > or a coroutine as argument); > > - either of the above, which is actually the most common requirement -- most > asyncio functions that support one also support the other, and either is > allowable as the argument to yield from. > > In the implementation we so often flipped between Future and coroutine that > I imagine sometimes the implementation and docs differ; also, we don't have > a good short name for "either of the above" so we end up using one or the > other as a shorthand. OK, that makes a lot of sense. > Unless you want to attach callbacks, inspect the result or exception, or > cancel it (all of which require a Future), your code shouldn't be concerned > about the difference -- you should just use res = yield from func(args) > and use try/except to catch exceptions if you care. And if you do need a > Future, you can call the function asyncio.async() on it (which in PEP 492 is > renamed to ensurefuture()). Again, makes sense. Although there are some bits of example code in the docs that call asyncio.async() on a coroutine and throw away the result (for example, https://docs.python.org/3/library/asyncio-task.html#example-future-with-run-until-complete ). That confuses me. Are you saying that async() modifies its (coroutine) argument to make it a Future? Rather than wrapping a coroutine in a Future, which gets returned?

No, it wraps a coroutine (i.e. a generator) in a Task, but leaves a Future alone. I'm stumped why that example calls async() and then throws the result away. I suspect it won't work without it (or else Victor wouldn't have added the call) but the reason seems, um, deep. I think wrapping it in a Task enters the generator in the event loop's queue of runnables -- otherwise the generator may well be garbage-collected without ever running.

Such complexity doesn't belong in such a simple example though.

> In the PEP 492 world, these concepts map as follows: > > - Future translates to "something with an await method" (and asyncio > Futures are trivially made compliant by defining Future.await as an > alias for Future.iter); > > - "asyncio coroutine" maps to "PEP 492 coroutine object" (either defined > with async def or a generator decorated with @types.coroutine -- note that > @asyncio.coroutine incorporates the latter); > > - "either of the above" maps to "awaitable".

OK. Although "future" is a nicer term than "something with an await method" and the plethora of flavours of coroutine is not great. But given that the only term we'll need in common cases is "awaitable", it's still a net improvement. So in the PEP 492 world, there's no such thing as a Task outside of asyncio? Or, to put it another way, a Task is only relevant in an IO context (unless an alternative event loop library implemented a similar concept), and we should only be talking in terms of awaitables and futures (given concurrent.futures and asyncio, I doubt you're going to be able to stop people using "Future" for the generic term for "something with an await method" at best, and quite possibly as equivalent to "awaitable", unfortunately).

I'm not sure. But it's true that Futures and Tasks in asyncio serve the purpose of linking the event loop (whose basic functioning is callback-based) to coroutines (implemented by generators). The basic idea is that when some I/O completes the event loop will call a callback function registered for that particular I/O operation; the callback then is actually a bound method of a Future or Task that causes the latter to be marked as "complete" (i.e. having a result) which in turn will call other callbacks (registered with the Future using add_done_callback()); in the case of a Task (i.e. a special kind of Future that wraps a generator/coroutine) this will resume the coroutine. (Actually it may resume an entire stack of coroutines that are blocked waiting for each other at yield-from; in my spare time I'm working on an explanation of the machinery underlying yield, yield from and await that will explain this.)

It's likely that you could write a much simpler event loop by assuming just coroutines (either the kind implemented by generators using yield from or the PEP 492 kind). The reason asyncio uses callbacks at the lower levels is the hope of fostering interoperability with Twisted and Tornado (and even gevent, which also has an event loop at the bottom of everything).

-- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20150505/9981fdaa/attachment.html>

More information about the Python-Dev mailing list