[Python-Dev] Re: Minimal 'stackless' PEP using generators? (original) (raw)

Terry Reedy tjreedy at udel.edu
Tue Aug 24 02:33:09 CEST 2004

"Clark C. Evans" <cce at clarkevans.com> wrote in message news:20040823153923.GA50408 at prometheusresearch.com...

I just read the thread 'Stackless Python' in June 2004 on python-dev and was wondering if you'd comment

Sure

on a simpler cooperative mechanism, via a small hack to generators:

If it is possible, as proposed, maybe at least a medium hack?

1. The PEP would introduce a new 'builtin' class called 'Cooperate'

2. Generator semantics would be altered so that 'yield X', where X is an instance of Cooperate, would automagically propigate to the outer-most non-generator.

-1

Skipping over the generic objections of 'hard to teach to beginners' and 'opens to door to a hundred other such specialized features':

  1. This alters and complexifies the current generator concept. At present, 'generator' (the return type of any generator function) is an instance of the iterator concept, as is 'iterator' (the return type of iter()) and any class with appropriate .iter and .next methods. A generator .next method is a resumable function; yield is like return except not permanent. The local namespace is not destroyed and instruction pointer not reset for the next call.

From the outside, a generator .next method is, I believe, indistinguishable in Python (excluding possible implementation-specific voodoo) from an interator-type .next method and behaviorally indistingueshable from any other iterator-protocol parameterless .next method. As far as I know, the difference between a generator and other iterators is ease of writing (sometimes much easier) and speed of execution, not external behavior.

  1. This imposes an isinstance(x, magic.Collaborate) function call cost on every yield in every generator. (I do not believe the alternative of imposing the same cost on every function call in every generator will work since I do not believe that callers can dependably know whether objects are returned or yielded. See comments on example.)

  2. 'automagic' it is. The pertinent runtime call stack contains not generator objects but .next methods (or in CPython, method-wrappers). So I presume you are proposing that yielded Collaborates magically plunk themselves into the first frame up the stack not associated such objects.

  3. If the returned to frame does not re-call the passed over suspended frames, they and anything they hold onto are leaked until gc'ed.

>>> def MyCooperate(magic.Cooperate): >>> str(self): return "cooperate" >>> >>> def top(): >>> yield "one" >>> yield MyCooperate() >>> yield "two" >>> >>> def middle(): >>> """ intermediate generator only sees one and two """ >>> for x in top(): >>> print "middle", x >>> yield x

Here, the effect you seek could be accomplished by an explicit 'if not isinstance(x, Collaborate):' before the print statement (or, in general, the block of code that uses x). But what if top is rewritten as def top(): return ("one", MyCooperate(), "two") Do you really want the behavior of middle to change, as with your proposal?

>>> def lower(): >>> """ this is not a generator, so it sees cooperate """ >>> for x in middle(): >>> print "lower", x

Now add 'if isinstance(x, Cooperate): return' and the suspended middle will be left in limbo. Not very tidy ;-)

Terry J. Reedy

More information about the Python-Dev mailing list