[Python-Dev] PEP 492: What is the real goal? (original) (raw)

Paul Moore p.f.moore at gmail.com
Thu Apr 30 20:52:46 CEST 2015

On 29 April 2015 at 20:19, Paul Moore <p.f.moore at gmail.com> wrote:

However, just to make my point precise, here's a more or less direct translation of the Wikipedia code into Python. It doesn't actually work, because getting the right combinations of yield and send stuff is confusing to me. Specifically, I suspect that "yield produce.send(None)" isn't the right way to translate "yield to produce". But it gives the idea.

Hmm, when I try to fix this "minor" (as I thought!) issue with my code, I discover it's more fundamental. The error I get is

Traceback (most recent call last): File ".\coro.py", line 28, in next(produce) File ".\coro.py", line 13, in produce yield consume.send(None) File ".\coro.py", line 23, in consume yield produce.send(None) ValueError: generator already executing

What I now realise that means is that you cannot have producer send to consumer which then sends back to producer. That's what the "generator already executing" message means.

This is fundamentally different from the "traditional" use of coroutines as described in the Wikipedia article, and as I thought was implemented in Python. The Wikipedia example allows two coroutines to freely yield between each other. Python, on the other hand, does not support this - it requires the mediation of some form of "trampoline" controller (or event loop, in asyncio terms) to redirect control. [1]

This limitation of Python's coroutines is not mentioned anywhere in PEP 342, and that's probably why I never really understood Python coroutines properly, as my mental model didn't match the implementation.

Given that any non-trivial use of coroutines in Python requires an event loop / trampoline, I begin to understand the logic behind asyncio and this PEP a little better. I'm a long way behind in understanding the details, but at least I'm no longer completely baffled.

Somewhere, there should be an explanation of the difference between Python's coroutines and Wikipedia's - I can't be the only person to be confused like this. But I don't think there's any docs covering "coroutines in Python" outside of PEP 342 - the docs just cover the components (the send and throw methods, the yield expression, etc). Maybe it could be covered in the send documentation (as that's what gives the "generator already executing" error. I'll try to work up a doc patch. Actually, looking at the docs, I can't even find where the behaviour of the send method is defined - can someone point me in the right direction?

Paul

[1] It's sort of similar to how Python doesn't do tail call elimination. Symmetric yields rely on stack frames that are no longer needed being discarded if they are to avoid unlimited recursion, so to have symmetric yields, Python would need a form of tail call ("tail yield", I guess :-)) elimination.

More information about the Python-Dev mailing list