(original) (raw)

On 27 Nov 2014 06:35, "Guido van Rossum" <guido@python.org> wrote:
\>
\> On Wed, Nov 26, 2014 at 3:24 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
\>> After thinking about that concern for a while, I'd like to suggest the
\>> idea of having a new builtin "allow\_implicit\_stop" decorator that
\>> swaps out a GENERATOR code object that has the new "EXPLICIT\_STOP"
\>> flag set for one with it cleared (attempting to apply
\>> "allow\_implicit\_stop" to a normal function would be an error).
\>>
\>> Then the updated version of the above example would become:
\>>
\>> @allow\_implicit\_stop
\>> def my\_generator():
\>> ...
\>> yield next(it)
\>> ...
\>>
\>> Which would be semantically equivalent to:
\>>
\>> def my\_generator():
\>> try:
\>> ...
\>> yield next(it)
\>> ...
\>> except StopIteration
\>> return
\>>
\>> but \*much\* faster (especially if used in a producer/consumer pipeline)
\>> since it would allow a single StopIteration instance to propagate
\>> through the entire pipeline, rather than creating and destroying new
\>> ones at each stage.
\>
\>
\> I think we can put a number to "much faster" now -- 150 nsec per try/except.
\>
\> I have serious misgivings about that decorator though -- I'm not sure how viable it is to pass a flag from the function object to the execution (which takes the code object, which is immutable) and how other Python implementations would do that. But I'm sure it can be done through sheer willpower. I'd call it the @hettinger decorator in honor of the PEP's most eloquent detractor. :-)

I agree with everything you wrote in your reply, so I'll just elaborate a bit on my proposed implementation for the decorator idea.

What I was thinking is that to implement the \_\_future\_\_ import, we're going to need a code object flag. For the sake of discussion, let's assume that flag is called "EXPLICIT\_STOP". When StopIteration escapes from a generator frame, we'll check for that flag and if it's set, StopIteration will be converted to RuntimeError. The details will differ for other implementations, but they're going to need something at least approximately equivalent to handle the transition period.

The implicit stop decorator would then check the flags on the code object attached to the passed in function. If GENERATOR wasn't set, that would be an immediate ValueError, while if EXPLICIT\_STOP wasn't set, the generator function would be passed through unmodified. However, if EXPLICIT\_STOP \*was\* set, the generator function would be replaced by a \*new\* generator function with a \*new\* code object, where the only change was to clear the EXPLICIT\_STOP flag.

The core part of the idea is actually keeping the runtime check for the EXPLICIT\_STOP flag on generator code objects even \*after\* setting the flag becomes the default compile time behaviour. The builtin (or itertools?) decorator to revert to the current behaviour on a per-generator basis then becomes a convenient way to access (and document!) that functionality without messing about with your own dynamic code object creation, and without coming up with new syntax for it (since new syntax can't be emulated on older versions of Python, or made implicit in other decorators like contextlib.contextmanager).

Regards,
Nick.