| msg343756 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2019-05-28 10:15 |
| Python 3.8 now has 3 hooks for uncaught exceptions: * sys.excepthook() * sys.unraisablehook() * threading.excepthook() _thread.start_new_thread() calls none of these hooks, but directly logs the exception. I propose to modify it to reuse sys.unraisablehook(): see attached PR. -- Using threading.excepthook() would be another option, but _thread.start_new_thread() is usually wrapped in threading.Thread.run() which already uses threading.excepthook(). It might be surprising to see two bugs from the same thread using threading.excepthook(). Moreover, right now, _thread is the "low-level" API to access threads: it doesn't acces threading. I'm not comfortable by adding an inter-dependency between _thread (low-level) and threading (high-level). If threading.Thread.run() is correctly written, it should not raise an exception. In the worst case, threading.excepthook should handle the exception. _thread.start_new_thread() should never get an exception raised by threading if threading.excepthook does correctly its job. |
|
|
| msg343757 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-05-28 10:19 |
| With PR 13617, the function which raised the exception can be retrieved using unraisable.object. If we used threading.excepthook, I don't see how we could pass this function into threading.ExceptHookArgs: ExceptHookArgs.thread is expected to be a threading.Thread object. |
|
|
| msg343759 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-05-28 10:23 |
| This issue is a follow-up of: * bpo-36829: Add sys.unraisablehook() to custom how "unraisable exceptions" are logged * bpo-1230540: Add threading.excepthook() to handle uncaught exceptions raised by Thread.run() Discussions around _thread.start_new_thead() exception: Antoine Pitrou: """ From the doc: Handle uncaught :func:``Thread.run`` exception. What if some C extension wants to report uncaught exceptions in C-created threads? Does it have to create a dummy Thread object? For example, how about threads created by _thread.start_new_thread? """ https://github.com/python/cpython/pull/13515#issuecomment-495935350 Serhiy Storchaka: "If we want to support low-level threads created by start_new_thread() we should call [threading.]excepthook() from t_bootstrap instead of Thread._bootstrap_inner. Otherwise implementing excepthook() as a Thread method would be more convenient." https://bugs.python.org/issue1230540#msg343748 |
|
|
| msg343788 - (view) |
Author: Christoph Reiter (lazka) * |
Date: 2019-05-28 15:16 |
| > _thread.start_new_thread() calls none of these hooks, but directly logs the exception. It calls sys.excepthook() currently: import _thread import threading import sys done = False def hook(*args): global done print(threading.current_thread()) done = True sys.excepthook = hook def worker(): raise Exception _thread.start_new_thread(worker, tuple()) while not done: pass |
|
|
| msg343805 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-05-28 16:44 |
| Output: --- Unhandled exception in thread started by <function worker at 0x7fae27f212d0> <_DummyThread(Dummy-1, started daemon 140385971111680)> --- Ah right, sys.excepthook is called! But "Unhandled exception in thread started by <function worker at 0x7fae27f212d0>" line is always written into stderr, it cannot be avoided :-( And sys.excepthook doesn't provide access to the thread function which caused the issue. |
|
|
| msg343806 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-05-28 16:49 |
| I updated my PR description: _thread.start_new_thread() now logs uncaught exception raised by the function using sys.unraisablehook(), rather than sys.excepthook(), so the hook gets access to the function which raised the exception. |
|
|
| msg343815 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2019-05-28 20:05 |
| > If threading.Thread.run() is correctly written, it should not raise an exception. Since the error handling for threading.Thread.run() is written on Python there are many opportunities to get an exception in error handling: KeyboardInterrupt, MemoryError and, at the shutdown stage, maybe NameError, AttributeError or TypeError. |
|
|
| msg343823 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-05-28 22:14 |
| > Since the error handling for threading.Thread.run() is written on Python there are many opportunities to get an exception in error handling: KeyboardInterrupt, MemoryError and, at the shutdown stage, maybe NameError, AttributeError or TypeError. Sure, the risk is real, but I tried to minimize it. NameError and AttributeError "should" not happen: I wrote _make_invoke_excepthook() to prevent name errors. Functions used to invoke threading.excepthook are "cached" in a private namespace. Moreover, the default hook is implemented in C also to reduce the risk of raising a new exception. Anyway, if threading.excepthook() raises a second exception, sys.excepthook() is now called to handle it ;-) That's a Python 3.8 change. In Python 3.7 and older, new exception was handled by start_new_thread(). |
|
|
| msg343845 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-05-29 00:58 |
| New changeset 8b09500345d998f3ff1e363a5210bc87f42ff306 by Victor Stinner in branch 'master': bpo-37076: _thread.start_new_thread() calls _PyErr_WriteUnraisableMsg() (GH-13617) https://github.com/python/cpython/commit/8b09500345d998f3ff1e363a5210bc87f42ff306 |
|
|
| msg343846 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-05-29 01:03 |
| Thanks Serhiy for the review! |
|
|