| msg232081 - (view) |
Author: Felipe (fov) * |
Date: 2014-12-03 17:51 |
| This bug report is the opposite of issue #14718. The interpreter did not raise an error when it encountered a `yield` expression inside the `finally` part of a `try/finally` statement. My system's info: Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:15:05) [MSC v.1600 32 bit (Intel)] on win32 More detail ======================================================= My interpreter does not raise any errors when yielding from a `finally` block. The documentation states, "Yield expressions are allowed in the try clause of a try ... finally construct."[1] Though not explicitly stated, the suggestion is that `yield` is not allowed in the `finally` clause. Issue #14718 suggests that this interpretation is correct. Not raising an exception for `yield` inside of `finally` can cause incorrect behavior when the generator raises its own unhandled exception in the `try` block. PEP 342 says, "If the generator raises an uncaught exception, it is propagated to send()'s caller." In this case, however, the exception gets paused at the `yield` expression, instead of propagating to the caller. Here's an example that can clarify the issue: >>> def f(): # I expected this function not to compile ... try: ... raise ValueError ... finally: ... yield 1 # Execution freezes here instead of propagating the ValueError ... yield "done" ... >>> g = f() >>> g.send(None) # PEP 342 would require ValueError to be raised here 1 >>> g.send(1) Traceback (most recent call last): File "", line 1, in File "", line 3, in f2 ValueError I may be arguing over the meaning of "uncaught exception," but I expected (given that the function compiles and doesn't raise a `RuntimeError`) the `ValueError` to propagate rather than get frozen. Example 2 ------------------------------------------------------- The second example is adapted from http://bugs.python.org/issue14718, where the submitter received a `RuntimeError`, but I did not: >>> def f2(): # I also expected this function not to compile ... try: ... yield 1 ... finally: ... yield 4 ... >>> g = f() # issue 14718 suggests this should raise RuntimeError >>> next(g) 1 >>> next(g) 4 >>> next(g) Traceback (most recent call last): File "", line 1, in StopIteration Possible resolution: ========================================================= 1. Enforce the ban on `yield` inside a finally `clause`. It seems like this should already be happening, so I'm not sure why my version isn't performing the check. This could be a run-time check (which seems like it may already be implemented), but I think this could even be a compile-time check (by looking at the AST). 2. Clarify the documentation to make explicit the ban on the use of `yield` inside the `finally` clause, and specify what type of error it will raise (i.e., `SyntaxError` or `RuntimeError`? or something else?). I'll submit a patch for 2 if there's support for this change, and I will work on 1 if someone can point me in the right direction. I've engaged with the C source relating to the different protocols, but have never looked through the compiler/VM. Notes ============================================ [1] https://docs.python.org/3.4/reference/expressions.html#yield-expressions |
|
|
| msg232082 - (view) |
Author: Guido van Rossum (gvanrossum) *  |
Date: 2014-12-03 18:38 |
| There is no prohibition in the language against yield in a finally block. It may not be a good idea, but the behavior you see is all as it should be. |
|
|
| msg232084 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2014-12-03 18:49 |
| FIlipe, in case you hadn't noticed, the reason for the error in the other issue is that the generator was closed when it got garbage collected, and it ignored the close (executed a yield after the close). Thus the error message there is accurate and expected, just as (as Guido said) the behavior here is consistent with the combined semantics of generators and try/finally. Do you want to suggest an improvement to the docs? It may be that we'll decide a change is more confusing than helpful, but we'll need someone to suggest a wording to decide that. |
|
|
| msg232086 - (view) |
Author: Ethan Furman (ethan.furman) * |
Date: 2014-12-03 19:05 |
| Here's the excerpt from the docs: -------------------------------- > Yield expressions are allowed in the try clause of a try ... finally construct. > If the generator is not resumed before it is finalized (by reaching a zero reference > count or by being garbage collected), the generator-iterator’s close() method will be > called, allowing any pending finally clauses to execute. This certainly makes it sound like 'yield' is okay in the 'try' portion, but not the 'finally' portion. So 'yield' is allowed in the 'try' portion, and it's allowed in the 'finally' portion -- is it also allowed in the 'except' portion? If so, we could simplify the first line to: > Yield expressions are allowed in try ... except ... finally constructs. |
|
|
| msg232087 - (view) |
Author: Felipe (fov) * |
Date: 2014-12-03 19:37 |
| Thanks for the clarification; sorry I misread issue #14718. I agree with Ethan's point. Though I would say "Yield expressions are allowed anywhere in try ... except ... finally constructs." I'd also like to explicitly add a point about the exception-handling machinery getting frozen, but I'm not sure how to phrase it clearly and accurately. Here's an attempt (my adds in square brackets): "By suspended, we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, the internal evaluation stack, [active exception handlers, and paused exceptions in finally blocks]." Another approach would be: "By suspended, we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack. [The state of any exception-handling code is also retained when yielding from a try ... except ... finally statement.]" |
|
|
| msg234058 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2015-01-15 06:26 |
| New changeset 0e48b0908651 by Ethan Furman in branch '3.4': Issue22988: clarify yield and exception blocks https://hg.python.org/cpython/rev/0e48b0908651 New changeset 55ba19d084fb by Ethan Furman in branch 'default': Issue22988: clarify yield and exception blocks https://hg.python.org/cpython/rev/55ba19d084fb |
|
|