[Python-ideas] Possible PEP 380 tweak (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Wed Oct 27 00:14:14 CEST 2010

On Wed, Oct 27, 2010 at 3:33 AM, Guido van Rossum <guido at python.org> wrote:

On Tue, Oct 26, 2010 at 7:44 AM, Jacob Holm <jh at improva.dk> wrote:

A different way to handle this would be to change the PEP 380 expansion as follows:

[...] - except GeneratorExit as e: + except (GeneratorReturn, GeneratorExit) as e: [...] That just strikes me as one more reason why a separate GeneratorReturn is a bad idea. In my ideal world, you almost never need to catch or raise StopIteration; you don't raise GeneratorExit (that is close()'s job) but you catch it to notice that your data source is finished, and then you return a value. (And see my crazy idea in my previous post to get rid of that too. :-)

Jacob's "implications for PEP 380" exploration started to give me some doubts, but I think there are actually some flaws in his argument. Accordingly, I would like to make one more attempt at explaining why I think throwing in a separate exception for this use case is valuable (and doesn't require any changes to PEP 380).

As I see it, there's a bit of a disconnect between many PEP 380 use cases and any mechanism or idiom which translates a thrown in exception into an ordinary StopIteration. If you expect your thrown in exception to always terminate the generator in some fashion, adopting the latter idiom in your generator will make it potentially unsafe to use in a "yield from" expression that isn't the very last yield operation in any outer generator.

Consider the following:

def example(arg): try: yield arg except GeneratorExit return "Closed" return "Finished"

def outer_ok1(arg): # close() after next() returns "Closed" return yield from example(arg)

def outer_ok2(arg): # close() after next() returns None yield from example(arg)

def outer_broken(arg): # close() after next() gives RuntimeError val = yield from example(arg) yield val

All 3 cases: close() before next() returns None

All 3 cases: close() after 2x next() returns None

Using close() to say "give me your return value" creates the risk of hitting those runtime errors in a generator's del method, and exceptions in del are always a bit ugly.

Keeping the "give me your return value" and "clean up your resources" concerns separate by adding a new method and thrown exception means that close() is less likely to unpredictably raise RuntimeError (and when it does, will reliably indicate a genuine bug in a generator somewhere that is suppressing GeneratorExit).

As far as PEP 380's semantics go, I think it should ignore the existence of anything like GeneratorReturn completely. Either one of the generators in the chain will catch the exception and turn it into StopIteration, or they won't. If they convert it to StopIteration, and they aren't the last generator in the chain, then maybe what actually needs to happen at the outermost level is something like this:

class GeneratorReturn(Exception): pass

def finish(gen): try: gen.throw(GeneratorReturn) # Ask generator to wrap things up except StopIteration as err: if err.args: return err.args[0] except GeneratorReturn: pass else: # Asking nicely didn't work, so force resource cleanup # and treat the result as if the generator had already # been exhausted or hadn't started yet gen.close() return None

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-ideas mailing list