Issue 14963: Use an iterative implementation for contextlib.ExitStack.exit (original) (raw)

The current implementation of contextlib.ExitStack [1] actually creates a nested series of frames when unwinding the callback stack in an effort to ensure exceptions are chained correctly, just as they would be if using nested with statements.

It would be nice to avoid this overhead by just using the one frame to iterate over the callbacks and handling correct exception chaining directly. This is likely to be a little tricky to get right, though, so the first step would be to set up a test that throws and suppresses a few exceptions and ensures the chaining when using ExitStack matches that when using nested with statements.

[1] http://hg.python.org/cpython/file/94a5bf416e50/Lib/contextlib.py#l227

Sorry, I wasn't clear on what I meant by "chained correctly", and that's the part that makes this trickier than the way contextlib.nested did it. I'm referring to the context attribute on exceptions that is set automatically when an exception occurs in another exception handler, which makes error displays like the following possible:

from contextlib import ExitStack with ExitStack() as stack: ... @stack.callback ... def f(): ... 1/0 ... @stack.callback ... def f(): ... {}[1] ... Traceback (most recent call last): File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 243, in _invoke_next_callback suppress_exc = _invoke_next_callback(exc_details) File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 240, in _invoke_next_callback return cb(*exc_details) File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 200, in _exit_wrapper callback(*args, **kwds) File "", line 7, in f KeyError: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 5, in File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 256, in exit return _invoke_next_callback(exc_details) File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 245, in _invoke_next_callback suppress_exc = cb(*sys.exc_info()) File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 200, in _exit_wrapper callback(*args, **kwds) File "", line 4, in f ZeroDivisionError: division by zero

The recursive approach maintains that behaviour automatically because it really does create a nested set of exception handlers. With the iterative approach, we leave the exception handler before invoking the next callback, so we end up bypassing the native chaining machinery and will need to recreate it manually.

If you can make that work, then we'd end up with the best of both worlds: the individual exceptions would be clean (since they wouldn't be cluttered with the recursive call stack created by the unwinding process), but exception chaining would still keep track of things if multiple exceptions are encountered in cleanup operations.

Interesting - it turns out we can't fully reproduce the behaviour of nested with statements in ExitStack (see the new reference test I checked in, as well as #14969)

I added one technically redundant variable to the implementation to make it more obviously correct to the reader, as well as a test that ensures the stack can handle ridiculous numbers of callbacks without failing (a key advantage of using a single frame rather than one frame per callback)

While it isn't mandatory, we prefer it if contributors submit Contributor Agreements even for small changes. If you're happy to do that, I consider emailing a scanned or digitally photographed copy of the signed form as described here to be the simplest currently available approach: http://www.python.org/psf/contrib/