[Python-Dev] async/await in Python; v2 (original) (raw)

Arnaud Delobelle arnodel at gmail.com
Sun Apr 26 10:48:49 CEST 2015

On 25 April 2015 at 22:02, Yury Selivanov <yselivanov.ml at gmail.com> wrote: [...]

On 2015-04-25 4:47 PM, Arnaud Delobelle wrote: [...]

1. About the 'async for' construct. Each iteration will create a new coroutine object (the one returned by Cursor.anext()) and it seems to me that it can be wasteful. In the example given of an 'aiterable' Cursor class, probably a large number of rows will fill the cursor buffer in one call of cursor.prefetch(). However each row that is iterated over will involve the creation execution of a new coroutine object. It seems to me that what is desirable in that case is that all the buffered rows will be iterated over as in a plain for loop.

I agree that creating a new coroutine object is a little bit wasteful. However, the proposed iteration protocol was designed to: 1. Resemble already existing iter/next/StopIteration protocol; 2. Pave the road to introduce coroutine-generators in the future.

Do you mean that aiter() would return a 'coroutine-generator'? I'm not sure what such an object is but if it is a suspendable generator in the same way that a coroutine is a suspendable function, then this is a strong argument to make the aiter() magic method a coroutine rather than a plain function. I.e. aiter() would return either an 'aiterator' or a 'coroutine generator object'. I think this could be mentioned in the section 'Why "aiter" is a coroutine' [1].

We could, in theory, design the protocol to make anext awaitable return a regular iterators (and raise StopAsyncIteration at the end) to make things more efficient, but that would complicate the protocol tremendously, and make it very hard to program and debug.

My opinion is that this has to be addressed in 3.6 with coroutine-generators if there is enough interest from Python users.

True, but to me this is bound to happen. I feel like the semantics of anext() is tied to the behaviour of this yet to be defined coroutine generator object and that if it turns out that the natural bevaviour of a coroutine generator is not consistent with the semantics of anext() then it would be a shame. I must say I have no evidence that this will happen!

2. I think the semantics of the new coroutine objects could be defined more clearly in the PEP. Of course they are pretty obvious when you know that the coroutines are meant to replace asyncio.coroutine as described in [1]. I understand that this PEP is mostly for the benefit of asyncio, hence mainly of interest of people who know it. However I think it would be good for it to be more self-contained. I have often read a PEP as an introduction to a new feature of Python. I feel that if I was not familiar with yield from and asyncio I would not be able to understand this PEP, even though potentially one could use the new constructs without knowing anything about them. I agree. I plan to update the PEP with some new semantics (prohibit passing coroutine-objects to iter(), tuple() and other builtins, as well as using them in 'for .. in coro()' loops). I'll add a section with a more detailed explanation of coroutine-objects.

Great! Thanks,

-- Arnaud

PS: there's a slight asymmetry in the terminology between coroutines and generators. 'Generator functions' are to 'generators' what 'coroutines' are to 'coroutine objects', which makes it difficult to what one is talking about when referring to a 'coroutine generator'.

[1] https://www.python.org/dev/peps/pep-0492/#id52

More information about the Python-Dev mailing list