warnings.catch_warnings is not thread safe or async safe · Issue #128384 · python/cpython (original) (raw)

Bug report

Bug description:

This is an old issue but it has become more urgent to fix given free-threaded Python. A simple example to show how the current context manager can fail:

from concurrent.futures import ThreadPoolExecutor

import warnings

def test_warning():
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        warnings.warn("my warning", UserWarning)

tpe = ThreadPoolExecutor(max_workers=10)

threads = [tpe.submit(test_warning) for _ in range(100)]
[t.result() for t in threads]

Since test_warning() is running in separate threads, there is no consistent ordering of which context manager exits first. That means the filters value that gets restored is also not consistent (a race between the threads about which version of the filters value they will restore).

The context manager is also not friendly to async code. These issues have been reported before:

• warnings.catch_warnings is async-unsafe #91505
• warnings should use a ContextVar to manage filters/registry #81785
• warnings.catch_warnings is not thread-safe #50896

The fairly obvious fix to this issue is to use contextvars, similar to how the decimal module uses a context. This is both thread safe and friendly to async code.

The warnings module itself has some "shallow" thread safety issues but these are relatively easy to fix and no API changes are needed.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-128384: Add thread-safe context manager to "warnings" module #128300
• gh-128384: Add locking to warnings.py. #128386
• gh-128384: Use contextvar for catch_warnings() #128463
• gh-128384: Use a context variable for warnings.catch_warnings #130010
• Revert "gh-128384: Use a context variable for warnings.catch_warnings (gh-130010) #132601