warnings.catch_warnings is not thread safe or async safe #128384
Description
Bug report
Bug description:
This is an old issue but it has become more urgent to fix given free-threaded Python. A simple example to show how the current context manager can fail:
from concurrent.futures import ThreadPoolExecutor
import warnings
def test_warning():
with warnings.catch_warnings():
warnings.simplefilter("ignore")
warnings.warn("my warning", UserWarning)
tpe = ThreadPoolExecutor(max_workers=10)
threads = [tpe.submit(test_warning) for _ in range(100)]
[t.result() for t in threads]
Since test_warning() is running in separate threads, there is no consistent ordering of which context manager exits first. That means the filters value that gets restored is also not consistent (a race between the threads about which version of the filters value they will restore).
The context manager is also not friendly to async code. These issues have been reported before:
- warnings.catch_warnings is async-unsafe #91505
- warnings should use a ContextVar to manage filters/registry #81785
- warnings.catch_warnings is not thread-safe #50896
The fairly obvious fix to this issue is to use contextvars, similar to how the decimal module uses a context. This is both thread safe and friendly to async code.
The warnings module itself has some "shallow" thread safety issues but these are relatively easy to fix and no API changes are needed.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
- gh-128384: Add thread-safe context manager to "warnings" module #128300
- gh-128384: Add locking to warnings.py. #128386
- gh-128384: Use contextvar for catch_warnings() #128463
- gh-128384: Use a context variable for warnings.catch_warnings #130010
- Revert "gh-128384: Use a context variable for warnings.catch_warnings (gh-130010) #132601
Activity
gh-128384: Add locking to warnings.py. (gh-128386)
Merge branch 'thread_inherit_context' into pythongh-128384-warnings-c…
Merge branch 'thread_inherit_context' into pythongh-128384-warnings-c…
Merge branch 'main' into pythongh-128384-warnings-contextvar
Merge 'origin/main' into pythongh-128384-warnings-contextvar
Merge branch 'origin/main' into pythongh-128384-warnings-contextvar
Merge 'origin/main' into pythongh-128384-warnings-contextvar
Merge 'origin/main' into pythongh-128384-warnings-contextvar
gh-128384: Use a context variable for warnings.catch_warnings (gh-130010
nascheme commentedon Apr 14, 2025
Fixed by GH-130010. You have to set the
sys.flags.context_aware_warningsflag to ensure you get the new behavior. If you are using threads, you will want to setsys.flags.thread_inherit_contextas well.Revert "pythongh-128384: Use a context variable for warnings.catch_wa…
Merge branch 'main' into pythongh-128384-test-fix
gh-128384: Fix for unreliable warnings unit tests. (gh-132611)