Repository navigation
address context shutdown racy condition. - #3219
Conversation
Signed-off-by: Tomoya Fujita <fujita.tomoya@triorb.co.jp>
fujitatomoya
left a comment
There was a problem hiding this comment.
this is ABI compatibility change, so that we can put this to lyrical and downstream branches.
|
Tick the box to add this pull request to the merge queue (same as
|
|
Pulls: #3219 |
|
https://github.com/Mergifyio backport humble jazzy kilted |
✅ Backports have been createdDetails
Cherry-pick of b723878 has failed: To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally
Cherry-pick of b723878 has failed: To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally
Cherry-pick of b723878 has failed: To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally |
Signed-off-by: Tomoya Fujita <fujita.tomoya@triorb.co.jp> (cherry picked from commit b723878) # Conflicts: # rclcpp/src/rclcpp/context.cpp
Signed-off-by: Tomoya Fujita <fujita.tomoya@triorb.co.jp> (cherry picked from commit b723878) # Conflicts: # rclcpp/src/rclcpp/context.cpp
Signed-off-by: Tomoya Fujita <fujita.tomoya@triorb.co.jp> (cherry picked from commit b723878) # Conflicts: # rclcpp/src/rclcpp/context.cpp # rclcpp/test/rclcpp/test_context.cpp
Signed-off-by: Tomoya Fujita <fujita.tomoya@triorb.co.jp> (cherry picked from commit b723878) # Conflicts: # rclcpp/src/rclcpp/context.cpp # rclcpp/test/rclcpp/test_context.cpp
`test_subscription_lease_manager` sometimes crashes in CI right after all its tests pass. Our test fixtures stop ROS during program exit, and a recent rclcpp update (ros2/rclcpp#3219) made that unsafe: by then, memory rclcpp needs for shutdown has already been freed. This change stops ROS at the end of each test suite instead, while the program is still running. ## What changed - A new fixture base in the test helpers initializes rclcpp when a test suite starts and shuts it down when the suite ends, so shutdown always happens while `main()` is still running. - Every fixture that used the static init now inherits from that base, not just the lease manager ones. The others don't crash today, but they could as soon as a test adds an earlier ROS shutdown. - Per-test `ScopedRclcppInit` locals are left alone, since they're destroyed inside the test body. I reproduced the crash locally against rclcpp 16.0.21, where it hit within a few runs. With this change, the full suite passes and 600 stress runs of the lease manager test finished without a crash.
### Changelog None ### Docs None ### Description `smoke_test` intermittently exits with SIGSEGV (-11) on Jazzy and Kilted after every test passes and the bridge logs "Shutdown complete". It happens on `main` too. - **Cause:** recent rclcpp releases ([ros2/rclcpp#3219](ros2/rclcpp#3219), backported to Jazzy 28.1.22 and Kilted 29.5.11) track in-progress context shutdowns in a `thread_local` set. The test's `main()` returned without calling `rclcpp::shutdown()`, so the global context was shut down by its static destructor during `exit()`, after glibc had already destroyed the main thread's `thread_local` objects. `spinReturnsOnContextShutdown` had created that set by shutting down a context on the main thread, so the exit-time shutdown inserts into freed memory. Whether that crashes depends on what the allocator left there. - **Fix:** destroy the bridge and executor while ROS is still valid, then call `rclcpp::shutdown()` before returning. This matches the production node's `main()`, which isn't affected. - **Verification:** reproduced locally with core dumps showing this stack, plus a small rclcpp-only program. With freed memory forced to garbage, the crash happens nearly every run on Jazzy and Kilted before the fix and never after. An ASan+UBSan build reports nothing through exit.
Description
retarget #3217 to lyrical
Fixes # (issue)
Is this user-facing behavior change?
Did you use Generative AI?
Additional Information