[Python-Dev] Using async/await in place of yield expression (original) (raw)

Nathaniel Smith njs at pobox.com
Mon Nov 27 03:41:55 EST 2017

On Sun, Nov 26, 2017 at 9:33 PM, Caleb Hattingh <caleb.hattingh at gmail.com> wrote:

The PEP only says that await must return an iterator, but it turns out that it's also required that that iterator should not return any intermediate values.

I think you're confused :-). When the iterator yields an intermediate value, it does two things:

(1) it suspends the current call stack and returns control to the coroutine runner (i.e. the event loop) (2) it sends some arbitrary value back to the coroutine runner

The whole point of await is that it can do (1) -- this is what lets you switch between executing different tasks, so they can pretend to execute in parallel. However, you do need to make sure that your await and your coroutine runner are on the same page with respect to (2) -- if you send a value that the coroutine runner isn't expecting, it'll get confused. Generally async libraries control both the coroutine runner and the await method, so they get to invent whatever arbitrary convention they want.

In asyncio, the convention is that the values you send back must be Future objects, and the coroutine runner interprets this as a request to wait for the Future to be resolved, and then resume the current call stack. In curio, the convention is that you send back a special tuple describing some operation you want the event loop to perform [1], and then it resumes your call stack once that operation has finished. And Trio barely uses this channel at all. (It does transfer a bit of information that way for convenience/speed, but the main work of setting up the task to be resumed at the appropriate time happens through other mechanisms.)

What you observed is that the asyncio coroutine runner gets cranky if you send it an integer when it was expecting a Future.

Since most libraries assume that they control both await and the coroutine runner, they don't tend to give great error messages here (though trio does [2] ;-)). I think this is also why the asyncio docs don't talk about this. I guess in asyncio's case it is technically a semi-public API because you need to know how it works if you're the author of a library like tornado or twisted that wants to integrate with asyncio. But most people aren't the authors of tornado or twisted, and the ones who are already know how this works, so the lack of docs isn't a huge deal in practice...

-n

[1] https://github.com/dabeaz/curio/blob/bd0e2cb7741278d1d9288780127dc0807b1aa5b1/curio/traps.py#L48-L156 [2] https://github.com/python-trio/trio/blob/2b8e297e544088b98ff758d37c7ad84f74c3f2f5/trio/_core/_run.py#L1521-L1530

-- Nathaniel J. Smith -- https://vorpus.org

More information about the Python-Dev mailing list