[Python-Dev] bpo-36829: Add sys.unraisablehook() (original) (raw)

Victor Stinner vstinner at redhat.com
Thu May 16 17:04:49 EDT 2019

Le jeu. 16 mai 2019 à 17:56, Steve Dower <steve.dower at python.org> a écrit :

I really like this API, and I agree with Victor that we don't really need more than the exception info. For future expansion, we can pass in a different exception, no?

Sorry, I don't understand. I explained that we need more than (exc_type, exc_value, exc_tb).

"obj" is part of the C function PyErr_WriteUnraisable(). We have to pass it to the hook. Otherwise, how do you want to guess where the exception comes from? Without any context, it can be super painful to debug such exception :-(

That's why I said that I like Serhiy's idea of extending the API to allow to also pass an error message.

Unraisable exceptions are commonly raised during garbage collection or in finalizers. So it's commonly happens when you don't "expect" them :-)

I'm not even convinced we need the obj argument, or that it's a good idea - this is yet another place where it's likely to be dead/dying already. And what can you do with it? Resurrection seems like a really bad idea, as does diving into a custom repr. There's no useful recovery mechanism here that I'm aware of, so I'd be in favor of just passing through the exception and nothing else.

Well, first of all, I would advice to not keep "obj" alive after the execution of the hook. Keep repr(obj) if you want, but not a reference to the object. Same for the exception value, keeping it alive is a high risk of creating annoying reference cycles ;-)

--

In a finalizer, "obj" is not always the object being finalized.

Example: when an _asyncio.Future is finalized, loop.call_exception_handler() is called if the future wasn't consumed. If this call fails, PyErr_WriteUnraisable(func) is called. func isn't the future, but the call_exception_handler() method.

Example: on a garbage collection, callbacks of weak references are called. If a callback fails, PyErr_WriteUnraisable(callback) is called. It's not the collected object nor the weak reference.

It's also common that PyErr_WriteUnraisable(NULL) is called. In this case, you have no context where the exception comes from :-( For that, I experimented a custom hook which logs also the current Python stack. That's useful in many cases!

socket.socket finalizer is a case where the finalized object (the socket) is passed to the hook. IMHO it's fine to resurrect a socket in that case. Finalization isn't deallocation. The objet remains consistent. The finalization protocol ensures that the finalizer is only called once.

In practice, I wouldn't advice to resurrect objects :-) I would expect that a hook only calls repr(hook) and then forgets the object.

The current PyErr_WriteUnraisable() implementation does exactly that: it formats repr(obj) into sys.stderr.

If someone finds a case where PyErr_WriteUnraisable() can resurect an object that is freed just after the call, I would suggest to fix the code to not pass the object to PyErr_WriteUnraisable(). Using PEP 442 finalizers, I don't think that it should happen.

Victor

Night gathers, and now my watch begins. It shall not end until my death.

More information about the Python-Dev mailing list