[Python-Dev] Re: Minimal 'stackless' PEP using generators? (original) (raw)

Clark C. Evans cce at clarkevans.com
Tue Aug 24 03:54:53 CEST 2004

Thanks for responding Terry. I'm not sure if my exposition was clear.

On Mon, Aug 23, 2004 at 08:33:09PM -0400, Terry Reedy wrote: | > 1. The PEP would introduce a new 'builtin' class called 'Cooperate' | > | > 2. Generator semantics would be altered so that 'yield X', where X | > is an instance of Cooperate, would automagically propigate to | > the outer-most non-generator.

Suppose I have three 'frames', two generators and one function, the call stack looks like:

 g2
 g1
 fn

For this to occur, fn has called .next() on g1, which has called .next() on g2. The proposed behavior is that "yield magic.Collaborate" provides the value returned by g1.next() within fn. In particular, it would be equivalent as if every "val = g2.next()" in g1 was rewritten:

val = g2.next()
if isinstance(val, magic.Cooperate):
    yield val

One could do this in three ways: (a) by a convention above, this is done in twisted.flow, (b) by a bytecode hack, or (c) by modifying eval.c to have equivalent behavior.

| 1. This alters and complexifies the current generator concept.

Correct; it introduces something similar to a corountine.

| 2. This imposes an isinstance(x, magic.Collaborate) function call cost on | every yield in every generator.

Ok. So, in addition to causing unexpected results (one would have to check to see what sort of object to debug), it would also cause a performance hit.

| 3. 'automagic' it is. The pertinent runtime call stack contains not | generator objects but .next methods (or in CPython, method-wrappers). So I | presume you are proposing that yielded Collaborates magically plunk | themselves into the first frame up the stack not associated such objects.

Right, the first frame that is not a generator, aka, not having a yield statement. And yes, it is different semantics.

| 4. If the returned to frame does not re-call the passed over suspended | frames, they and anything they hold onto are leaked until gc'ed.

No. See the above equivalence. It might take a bit of thinking to make the "C" implementation correct, but the semantics are farily straight-forward and should not create leaks.

| Here, the effect you seek could be accomplished by an explicit 'if not | isinstance(x, Collaborate):' before the print statement (or, in general, | the block of code that uses x). But what if top is rewritten as | def top(): return ("one", MyCooperate(), "two") | Do you really want the behavior of middle to change, as with your proposal?

Yes.

| > >>> def lower(): | > >>> """ this is not a generator, so it sees cooperate """ | > >>> for x in middle(): | > >>> print "lower", x || Now add 'if isinstance(x, Cooperate): return' and the suspended middle will | be left in limbo. Not very tidy ;-)

No, it would be just like a generator had 'if 1: return'; the suspended middle would be cleaned up just as it is today.

...

However, you've convinced me on the semantic differences and the performance hit. I guess the proposal would have to present a new keyword, say 'collaborate' and have to have its semantics clearly specified.

Thank you so much for your kind review,

Clark

More information about the Python-Dev mailing list