[Python-Dev] Handle errors in cleanup code (original) (raw)
Nathaniel Smith njs at pobox.com
Tue Jun 13 04:43:54 EDT 2017
- Previous message (by thread): [Python-Dev] Handle errors in cleanup code
- Next message (by thread): [Python-Dev] Handle errors in cleanup code
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On Tue, Jun 13, 2017 at 12:10 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
reporting failures from concurrent.futures.wait: https://pythonhosted.org/futures/#concurrent.futures.wait
Yeah, and asyncio.gather is another example in the stdlib. Or there's twisted's DeferredList. Trio is unusual in effectively forcing all tasks to be run under a gather(), but the basic concept isn't unique at all.
Figuring out how to display an exception tree coherently is going to be a pain (it's already problematic with just the linked list), but if we can at least model exception trees consistently, then we'd be able to share that display logic, even if the scenarios resulting in MultiErrors varied.
It's true, it was a pain :-). And I'm sure it can be refined. But I think this is a reasonable first cut: https://github.com/python-trio/trio/blob/9e0df6159e55fe5e389ae5e24f9bbe51e9b77943/trio/_core/_multierror.py#L341-L388
Basically the algorithm there is:
if there's a cause or context: print it (recursively using this algorithm) print the description line ("this exception was the direct cause" or whatever is appropriate) print the traceback and str() attached to this exception itself for each embedded exception: print "Details of embedded exception {i}:" with extra indentation: print the embedded exception (recursively using this algorithm) (+ some memoization to avoid infinite recursion on loopy structures)
Of course really complicated trainwrecks that send exception shrapnel flying everywhere can still be complicated to read, but ... that's why we make the big bucks, I guess. (My fingers initially typoed that as "that's why we make the big bugs". Just throwing that out there.)
Currently that code has a hard-coded assumption that the only kind of exception-container is MultiError, but I guess it wouldn't be too hard to extend that into some kind of protocol that your asymmetric CleanupError could also participate in? Like: an abstract exception container has 0 or 1 (predecessor exception, description) pairs [i.e., cause or context], plus 0 or more (embedded exception, description) pairs, and then the printing code just walks the resulting tree using the above algorithm?
If this all works out at the library level, then one can start to imagine ways that the interpreter could potentially get benefits from participating:
right now, if an exception that already has a cause or context is re-raised, then implicit exception chaining will overwrite the old cause or context. Instead, it could check to see if there's already a non-None value there, and if so do exc.context = MultiError([exc.context, new_exc]).
standardizing the protocol for walking over an exception container would let us avoid fighting over who owns sys.excepthook, and make it easier for third-party exception printers (like IPython, pytest, etc.) to play along.
MultiError.catch is a bit awkward. If we were going all-in on this then one can imagine some construct like
multitry: ... do stuff ... raise MultiError([ValueError(1), MultiError([TypeError(), ValueError(2)])]) multiexcept ValueError as exc: print("caught a ValueError", exc) multiexcept TypeError: print("caught a TypeError and raising RuntimeError") raise RuntimeError
which prints
caught a ValueError 1 caught a TypeError and raising RuntimeError caught a ValueError 2
and then raises a RuntimeError with its context pointing to the TypeError.
...but of course that's preeeeetty speculative at this point; definitely more python-ideas territory.
-n
-- Nathaniel J. Smith -- https://vorpus.org <http://vorpus.org> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20170613/ec4ee64b/attachment.html>
- Previous message (by thread): [Python-Dev] Handle errors in cleanup code
- Next message (by thread): [Python-Dev] Handle errors in cleanup code
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]