Checking for sys.is_finalizing before thread.start() still yields "can't create new thread at interpreter shutdown" · Issue #114570 · python/cpython (original) (raw)
Bug report
Bug description:
We have code in pymongo Our MongoClient object immediately spawns a new daemon thread to run some periodic check operations from the moment it is initialized. When run on python 3.12, we've run into the issue can't create new thread at interpreter shutdown because we have cases thread.start() that can execute as the interpreter is shutting down.
To mitigate this issue we attempted checking sys.is_finalizing before issuing the start() call:
if not sys.is_finalizing(): thread.start()
This still generated the same runtime error.
We tried catching the RuntimeError exception:
try: thread.start() except RuntimeError: pass
This worked. This feels too broad a solution than the above since it will catch and ignore all RuntimeErrors.
Finally we decided to see if we could check sys.is_finalizing after the exception is raised.
try: thread.start() except RuntimeError: if sys.is_finalizing(): self._thread = None return raise
To our surprise, this also failed. I can understand that starting a thread on interpreter shutdown should be avoided, but this also feels misleading if the system isn't correctly seeing--which is my assumption on what sys.is_finalizing()--that it is experiencing interpreter shutdown.
Should the expectation not be that sys.is_finalizing is updated before the thread.start() is called OR by the time the RuntimeError is raised? This way we can safely catch or preempt the error before triggering?
As a general inquiry, though, what should be the general best practice for patching issues like this on our end as it has introduced somewhat breaking behavior from 3.11 -> 3.12? Should we do like above with catching the RuntimeError but additionally check the exception string for the exact type of RuntimeError?
CPython versions tested on:
3.12
Operating systems tested on:
macOS