[Python-Dev] PEP 334 - Simple Coroutines via SuspendIteration (original) (raw)

Samuele Pedroni pedronis at bluewin.ch
Wed Sep 8 20:58:21 CEST 2004

James Y Knight wrote:

On Sep 7, 2004, at 9:48 PM, Clark C. Evans wrote:

I've packaged up the idea of a coroutine facility using iterators and an exception, SuspendIteration. Very interesting. This proposal assumes that a corresponding iterator written using this class-based method is possible for existing generators. The challenge seems to be the identification of distinct states within the generator where suspension could occur. That is basically impossible. Essentially every operation could possibly raise SuspendIteration, because essentially every operation can call an arbitrary python function, and python functions can raise any exception they want. I think you could still make the proposal work in CPython: if I understand its internals properly, it doesn't need to do a transformation to a class iterator, it simply suspends the frame wherever it is. Thus, being able to suspend at any point in the function would not cause an undue performance degradation.

I don't think it is that simple for CPython either, a single bytecode can potentially invoke more then just a single builtin or other python code, e.g. calling construction can invoke new and init, and then there are all the cases were descriptors are involved with their get etc (and add,radd...). So bytecodes are not the right suspension/resumption granularity because you don't want to reinvoke things that could have had side-effects. So you have all the points per bytecode were python code/builtins can be invoked or from another POV an exception can be detected.

If I understand the proposal (which is quite vague), like restartable syscalls, there is also the matter that whatever raised the SuspendIteration should be retried on resumption of the generator, e.g calling nested generator next.

So one would have to cherry pick for each bytecode or similar abstract operations model relevant suspension/resumption points and it would still be quite a daunting task to implement this adding the code for intra bytecode resumption. (Of course this assumes that capturing the C stack and similar techniques are out of question)

However, I think it is a deal-breaker for JPython. From the generator PEP: "It's also believed that efficient implementation in Jython requires that the compiler be able to determine potential suspension points at compile-time, and a new keyword makes that easy." If this quote is right about the implementation of Jython (and it seems likely, given the JVM), your proposal makes it impossible to implement generators in Jython.

a hand coded implementation would be a lot of work (beyound practical) for potentially very bad performance and a resulting messy codebase. One could also encounter resulting code size problems or issues with the verifier.

More information about the Python-Dev mailing list