Skip to content

gh-158973: Reset the warnings lock after fork in the child - #158974

Merged
nascheme merged 3 commits into
python:mainfrom
nascheme:gh-158973-reset-warnings-lock
Oct 17, 2026
Merged

nascheme merged 3 commits into
python:mainfrom
nascheme:gh-158973-reset-warnings-lock

Conversation

@nascheme

@nascheme nascheme commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Reset the warnings lock in the child after fork, before clearing other threads’ states. This prevents deadlocks when their finalizers issue warnings and the lock was held by a thread that no longer exists. Record ownership before fork so the child can preserve the lock’s recursion depth and update its owner ID if the forking thread held it. Adds regression tests for child-cleanup warnings and recursive ownership across fork.

The PR was developed with assistance from GPT 6.1 Sol.

Add _PyRecursiveMutex_at_fork_reinit() helper function.
@nascheme
nascheme marked this pull request as ready for review October 9, 2026 17:22
@leandrodamascena

Copy link
Copy Markdown

I was able to reproduce this independently and validate the fix.

With a probe similar to test_fork_other_thread_holds_lock (a worker holds the warnings lock and keeps an object with a warning-emitting finalizer in thread-local storage, then the main thread forks), the child hangs inside os.fork() in every attempt on main before this PR. It stays stuck until it's killed, even with a 10 s timeout. With this PR, the child runs the finalizer, writes its marker and exits with status 0 every time, and the parent's worker releases the lock and joins cleanly.

before this PR with this PR
child hangs and is killed by the watchdog 100/100 0/100
child exits 0 and the finalizer runs 0/100 100/100

I saw the same on AWS Lambda (x86_64, PGO+LTO build), where this matters since the default runtime is a regular GIL build: every attempt hit the watchdog before the fix, and none did after it. test_warnings, test_fork1 and test_os pass with the change. Thanks for fixing this!

@ngoldbaum

Copy link
Copy Markdown
Contributor

I think it's safe to store the bool in a static global: https://github.com/nascheme/cpython/compare/gh-158973-reset-warnings-lock...ngoldbaum:cpython:gh-158973-reset-warnings-lock?expand=1

This is being maximally cautions about avoiding ABI changes where they're not needed. I'm not sure if it's actually a problem in-practice. The static global is a bit of a code smell.

@ngoldbaum ngoldbaum left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good to merge for main but I'm not sure about the backport, see my comment.

ngoldbaum added a commit to ngoldbaum/cpython that referenced this pull request Oct 16, 2026
ngoldbaum added a commit to ngoldbaum/cpython that referenced this pull request Oct 16, 2026
@nascheme
nascheme merged commit 8013779 into python:main Oct 17, 2026
151 of 155 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @nascheme for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @nascheme, I could not cleanly backport this to 3.14 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker 801377967fd7bfa573c0b4c48e31b69d50c1da2f 3.14

@bedevere-app

bedevere-app Bot commented Oct 17, 2026

Copy link
Copy Markdown

GH-159613 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 17, 2026
@bedevere-app

bedevere-app Bot commented Oct 17, 2026

Copy link
Copy Markdown

GH-159614 is a backport of this pull request to the 3.15 branch.

@bedevere-app

bedevere-app Bot commented Oct 17, 2026

Copy link
Copy Markdown

GH-159615 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-bug An unexpected behavior, bug, or error

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants