[Python-Dev] async/await in Python; v2 (original) (raw)

Yury Selivanov yselivanov.ml at gmail.com
Wed Apr 22 22:25:38 CEST 2015

Hi PJ,

On 2015-04-22 3:44 PM, PJ Eby wrote:

On Tue, Apr 21, 2015 at 1:26 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:

It is an error to pass a regular context manager without _aenter_ and _aexit_ methods to async with. It is a SyntaxError to use async with outside of a coroutine. I find this a little weird. Why not just have with and for inside a coroutine dynamically check the iterator or context manager, and either behave sync or async accordingly? Why must there be a syntactic difference?

One of the things that we try to avoid is to have implicit places where code execution might be suspended. For that we use 'yield from' right now, and want to use 'await' with PEP 492.

To have implicit context switches there is Stackless Python and greenlets, however, it's harder to reason about the code written in such a way. Having explicit 'yield from/await' is the selling point of asyncio and other frameworks that use generator-based coroutines.

Hence, we want to stress that 'async with' and 'async for' do suspend the execution in their protocols.

I don't want to loose control over what kind of iteration or context manager I'm using. I don't want to iterate through a cursor that doesn't do prefetching, I want to make sure that it does. This problem is solved by the PEP.

Not only would this simplify the syntax, it would also allow dropping the need for async to be a true keyword, since functions could be defined via "def async foo():" rather than "async def foo():" ...which, incidentally, highlights one of the things that's been bothering me about all this "async foo" stuff: "async def" looks like it defines the function asynchronously (as with "async with" and "async for"), rather than defining an asynchronous function. ISTM it should be "def async bar():" or even "def bar() async:".

If we keep 'async with', then we'll have to keep 'async def' to make it symmetric and easier to remember. But, in theory, I'd be OK with 'def async'.

'def name() async' is something that will be extremely hard to notice in the code.

Also, even that seems suspect to me: if await looks for an await method and simply returns the same object (synchronously) if the object doesn't have an await method, then your code sample that supposedly will fail if a function ceases to be a coroutine *will not actually fail*.

It doesn't just do that. In the reference implementation, a single 'await o' compiles to:

(o) # await arg on top of the stack GET_AWAITABLE LOAD_CONST None YIELD_FROM

Where GET_AWAITABLE does the following:

If you had a code like that:

await coro()

where coro is

async def coro(): pass

you then can certainly refactor core to:

def coro(): return future # or some awaitable, please refer to PEP492

And it won't break anything.

So I'm not sure I understand your remark about "will not actually fail".

In my experience working with coroutine systems, making a system polymorphic (do something appropriate with what's given) and idempotent (don't do anything if what's wanted is already done) makes it more robust. In particular, it eliminates the issue of mixing coroutines and non-coroutines.

Unfortunately, to completely eliminate the issue of reusing existing "non-coroutine" code, or of writing "coroutine" code that can be used with "non-coroutine" code, you have to use gevent-kind of libraries.

To sum up: I can see the use case for a new await distinguished from yield, but I don't see the need to create new syntax for everything; ISTM that adding the new asynchronous protocols and using them on demand is sufficient. Marking a function asynchronous so it can use asynchronous iteration and context management seems reasonably useful, but I don't think it's terribly important for the type of function result. Indeed, ISTM that the built-in object class could just implement _await_ as a no-op returning self, and then all results are trivially asynchronous results and can be awaited idempotently, so that awaiting something that has already been waited for is a no-op.

I see all objects implementing await returning "self" as a very error prone approach. It's totally OK to write code like that:

async def coro(): return fut future = await coro()

In the above example, if coro ceases to be a coroutine, 'future' will be a result of 'fut', not 'fut' itself.

(Prior art: the Javascript Promise.resolve() method, which takes either a promise or a plain value and returns a promise, so that you can write code which is always-async in the presence of values that may already be known.)

Finally, if the async for and with operations have to be distinguished by syntax at the point of use (vs. just always being used in coroutines), then ISTM that they should be with async foo: and for_ _async x in bar:, since the asynchronousness is just an aspect of how the main keyword is executed. tl;dr: I like the overall ideas but hate the syntax and type segregation involved: declaring a function async at the top is OK to enable async with/for semantics and await expressions, but the rest seems unnecessary and bad for writing robust code. (e.g. note that requiring different syntax means a function must either duplicate code or restrict its input types more, and type changes in remote parts of the program will propagate syntax changes throughout.)

Thanks, Yury

More information about the Python-Dev mailing list