Repository navigation
Fix fork-child hangs on copied condition variables (musl); skip re-testing identical trees on develop - #69
Merged
Conversation
…the parent (musl) A child of fork() kept the parent's copies of the module's condition variables, which still listed the parent's waiting threads as waiters. Those threads do not exist in the child, and musl's pthread_cond_signal()/broadcast() wait for the waiters that they wake, so the child hung (seen in test-alpine: standalone-signals.py hung in the child's first Qore call that released the GIL, in notify_one() of the GIL switch helper's condition variable while holding its lock). - gil_switch_fork_child(): construct GilSwitchState::cond again in place (the helper thread always waits on it while the prepare handler holds the lock) - forkChildThreadStates(): construct pgm_thr_cond of every interpreter again - QorePythonDeclHandle: a new child handler clears the threads waiting in invalidate(), constructs the condition again, and sets the use count of every handle to the forking thread's uses; previously the uses of the parent's other threads never ended in the child, and deleting a Program waited for them forever. All handles are kept in an intrusive list so the handler reaches them without allocating memory. The old objects are not destroyed, as pthread_cond_destroy() can also wait for the waiters (as QoreCondition::reinitAfterFork() in libqore, which modules cannot call). Tests: new test/standalone-fork.py (forks with the GIL switch helper idle and busy; the child waits for the GIL in Qore calls), a new python-fork.qtest case (the child deletes a Program whose class another thread was constructing; it hangs without the fix), and standalone-signals.py waits for its child with a deadline (pidfd) instead of waiting forever.
… tests As in module-ncurses (73d328c) and module-pgsql (251bac2): the new tree-already-tested job (.pre, develop pushes only) finds a successful pipeline for the commit or for a parent with the identical tree (the merged pull request branch) in which every test job passed, and each test job then reports success to GitHub under its own context with a link to that pipeline and ends before building. Missing or incomplete evidence runs every job in full; QORE_CI_FORCE_FULL=1 forces a full run. test/test_tree_tested.py tests the scripts and checks that the required jobs of tree-already-tested.py match the test jobs of .gitlab-ci.yml; it runs in every test job.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fork fix: On Alpine (musl),
test/standalone-signals.pycould hang forever in the child ofos.fork(). The child kept the parent's copy of the GIL switch helper's condition variable, which still listed the parent's helper thread as a waiter. musl'spthread_cond_signal()waits for the waiters it wakes, so the child'snotify_one()never returned while it held the helper's lock (jobtest-alpine228940, pipeline 64826).gil_switch_fork_child(): the child constructsGilSwitchState::condagain in place.QorePythonProgram::pgm_thr_cond, inforkChildThreadStates().pthread_cond_destroy()can also wait for waiters. This matchesQoreCondition::reinitAfterFork(), which modules can't call.CI: develop push pipelines now skip the test jobs when the identical tree already passed every test job, for example on the merged PR branch. This is ported from module-ncurses 73d328c and module-pgsql 251bac2.
QORE_CI_FORCE_FULL=1forces a full run, and missing or incomplete evidence also runs every job in full.Tests
test/standalone-fork.py: forks 20 times with the GIL switch helper idle and 20 times while it is busy. The children wait for the GIL in Qore calls.python-fork.qtestcase: the child deletes a Program whose class another thread was constructing. Without the fix it hangs; verified locally, killed bytimeout.standalone-signals.pyand the new test wait for children with a deadline, using a pidfd andselect, instead of waiting forever. A hung child now fails the test instead of holding the job until the CI timeout.test/test_tree_tested.py: 24 unit tests for the CI scripts.develop.test-alpinejob runs the new tests.Known gap: there is no test for a thread that waits in
invalidate()orwaitForThreadsIntern()at the moment of the fork. A test can't tell that a thread has reached that wait without polling, so those two fixes are covered by code review only.🤖 Generated with Claude Code