Skip to content

Fix/component manager service qos - #30

Closed
nago-7563 wants to merge 30 commits into
tier4:masterfrom
nago-7563:fix/component-manager-service-qos
Closed

nago-7563 wants to merge 30 commits into
tier4:masterfrom
nago-7563:fix/component-manager-service-qos

Conversation

@nago-7563

Copy link
Copy Markdown

Description

This PR increases the service queue depth of ~/_container/load_node and ~/_container/unload_node in ComponentManager from the default 10 to 200.

The same issue was reported upstream in ros2#2362 and fixed in ros2#2363 (backported to Humble in ros2#2380). The humble_tracepoint_added branch is based on 16.0.5 and does not include this fix (the fix is in ros2/rclcpp Humble since 16.0.8).
Instead of cherry-picking ros2#2380, the change is written to match the current code of rclcpp_components/src/component_manager.cpp on the ros2/rclcpp humble branch. For this part, that code is identical to ros2#2380. list_nodes is not changed, as in upstream.

Problem

With the default service QoS (KEEP_LAST, depth 10), when more than 10 LoadNode requests are pending in a single-threaded component container, the oldest requests are overwritten in the DDS reader history and silently dropped. The DDS layer has already ACKed them, so the client (launch_ros) waits forever and the node is never loaded.

In our environment (Autoware + AWSIM, ROS 2 Humble, CycloneDDS, built with CARET), component_state_monitor.launch.py sends 15 LoadNode requests in parallel to /system/component_state_monitor/container at startup. Under CPU load (e.g., with a limited number of cores or a lowered CPU frequency), some topic_state_monitor nodes were not loaded, and Autoware could not enter autonomous mode. Without CARET, the same setup works because the stock Humble rclcpp_components already has the fix.

We confirmed with a CycloneDDS discovery trace that the request reader of load_node has history=0:10 (KEEP_LAST, 10) before this change and history=0:200 after.

Reproduction

Same as ros2#2362: one component_container and 30 LoadComposableNodes of composition::Talker sent in parallel; count loaded nodes with ros2 component list after 15 s.

Environment Loaded nodes
CARET environment (setenv_caret.bash), before this change 17-18 / 30
Stock ROS 2 Humble (/opt/ros/humble) 30 / 30
CARET environment, with this change 30 / 30

With this change applied, the topic_state_monitor loading failure no longer occurs in our Autoware startup tests, including runs with restricted CPU resources.

Related

mergify Bot and others added 30 commits June 9, 2022 08:54
* Add statistics for handle_loaned_message

Signed-off-by: Barry Xu <barry.xu@sony.com>
(cherry picked from commit 5c68830)

Co-authored-by: Barry Xu <barry.xu@sony.com>
This fails with g++ -std=gnu++20.

Signed-off-by: Jochen Sprickerhof <git@jochen.sprickerhof.de>
(cherry picked from commit 02802bc)

Co-authored-by: Jochen Sprickerhof <github@jochen.sprickerhof.de>
Signed-off-by: hsgwa <hasegawa.isp@gmail.com>
Signed-off-by: hsgwa <hasegawa.isp@gmail.com>
* use regex for wildcard matching (ros2#1839)

* use regex for wildcard matching

Co-authored-by: Aaron Lipinski <aaron.lipinski@roboticsplus.co.nz>
Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* use map to process the content of parameter file by order

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* add more test cases

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* try to not decrease the performance and make the param win last

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* update node name

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* update document comment

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* add more test for parameter_map_from

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

Co-authored-by: Aaron Lipinski <aaron.lipinski@roboticsplus.co.nz>
(cherry picked from commit 6dd3a03)

* not to break ABI

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

Signed-off-by: Chen Lihui <lihui.chen@sony.com>
Co-authored-by: Chen Lihui <lihui.chen@sony.com>
* fix mismatched issue if uzing zero_allocated

Signed-off-by: Chen Lihui <lihui.chen@sony.com>
(cherry picked from commit 9784391)

Co-authored-by: Chen Lihui <lihui.chen@sony.com>
Signed-off-by: Audrow Nash <audrow@openrobotics.org>
* Add a test case

a subscriber on a new executor with a callback group triggered to receive a message

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* fix flaky and not to use spin_some

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* update comment

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* update for not using anti-pattern source code

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* add a notify guard condition for callback group

Co-authored-by: William Woodall <william@osrfoundation.org>
Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* Notify guard condition of Node not to be used in Executor

it is only for the waitset of GraphListener

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* put code in the try catch

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* defer to create guard condition

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* use context directly for the create function

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* cpplint

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* fix that some case might call add_node after shutdown

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* nitpick and fix some potential bug

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* add sanity check as some case might not create notify guard condition after shutdown

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* Cleanup includes.

Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>

* remove destroy method

make a callback group can only create one guard condition

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

* remove limitation that guard condition can not be re-created in callback group

Signed-off-by: Chen Lihui <lihui.chen@sony.com>

Signed-off-by: Chen Lihui <lihui.chen@sony.com>
Signed-off-by: Chris Lalancette <clalancette@openrobotics.org>
Co-authored-by: William Woodall <william@osrfoundation.org>
Co-authored-by: Chris Lalancette <clalancette@openrobotics.org>
(cherry picked from commit d119157)

Co-authored-by: Chen Lihui <lihui.chen@sony.com>
…2#2065)

* fix nullptr dereference in prune_requests_older_than

Signed-off-by: akela1101 <akela1101@gmail.com>

* add tests for prune_requests_older_than

Signed-off-by: akela1101 <akela1101@gmail.com>

* Update rclcpp/test/rclcpp/test_client.cpp

Co-authored-by: Chen Lihui <lihui.chen@sony.com>
Signed-off-by: akela1101 <akela1101@gmail.com>

Signed-off-by: akela1101 <akela1101@gmail.com>
Co-authored-by: Chen Lihui <lihui.chen@sony.com>
(cherry picked from commit 1ac37b6)

Co-authored-by: andrei <akela1101@gmail.com>
…ffer (ros2#2061) (ros2#2070)

Signed-off-by: Alberto Soragna <alberto.soragna@gmail.com>

Signed-off-by: Alberto Soragna <alberto.soragna@gmail.com>
(cherry picked from commit 3fb012e)

Co-authored-by: Alberto Soragna <alberto.soragna@gmail.com>
…) (ros2#2076)

Signed-off-by: Lei Liu <Lei.Liu.AP@sony.com>
(cherry picked from commit 66b1944)

Co-authored-by: Lei Liu <64953129+llapx@users.noreply.github.com>
Signed-off-by: Audrow Nash <audrow@openrobotics.org>
Signed-off-by: Tomoya Fujita <Tomoya.Fujita@sony.com>
(cherry picked from commit 1a796b5)

Co-authored-by: Tomoya Fujita <Tomoya.Fujita@sony.com>
backport of ros2#2132

Signed-off-by: Tomoya Fujita <Tomoya.Fujita@sony.com>
Co-authored-by: Chen Lihui <lihui.chen@sony.com>
…os2#2133)" (ros2#2148)

This reverts commit c8ac675.

Signed-off-by: Tomoya Fujita <Tomoya.Fujita@sony.com>
…sued. (ros2#2133)" (ros2#2148)" (ros2#2152)

This reverts commit 19a666f.

Signed-off-by: Tomoya Fujita <Tomoya.Fujita@sony.com>
Signed-off-by: Audrow Nash <audrow@intrinsic.ai>
(cherry picked from commit c091fe1)

Co-authored-by: methylDragon <methylDragon@gmail.com>
…2167)

If the intraprocess buffer still has data after taking, re-trigger the
guard condition to ensure that the executor will continue to service it,
even if incoming publications stop.

Signed-off-by: Michael Carroll <michael@openrobotics.org>
(cherry picked from commit 5f9695a)

Co-authored-by: Michael Carroll <michael@openrobotics.org>
…os2#2219) (ros2#2223)

ros2#2167 (comment)

Signed-off-by: Tomoya Fujita <Tomoya.Fujita@sony.com>
(cherry picked from commit fe2e0e4)

Co-authored-by: Tomoya Fujita <Tomoya.Fujita@sony.com>
…os2#2183)

* add initially-failing test case

* apply changes to LifecycleNodeInterfaceImpl from ros2#1756

* add static member to State for managing state_handle_ access

* allow parallel read access in MutexMap

Signed-off-by: Joe Schornak <joe.schornak@gmail.com>
Signed-off-by: Audrow Nash <audrow@intrinsic.ai>
* change publish tp def

Signed-off-by: yamasaki <114902604+ymski@users.noreply.github.com>

* added publisher_handle

Signed-off-by: yamasaki <114902604+ymski@users.noreply.github.com>

---------

Signed-off-by: yamasaki <114902604+ymski@users.noreply.github.com>
Merge commit 'ae8b033ae01601475104bf9e4c2350b98c9560c4' into work_for_update_humble_tracepoint_added
Merge commit '9171122eae940504200889b2a7b12344fa7c0c8b' into work_for_update_humble_tracepoint_added
Merge commit '00ef09cbf34563db27b9aac9b0206eb142cf3995' into work_for_update_humble_tracepoint_added
Merge commit '52327dd3a3b0fc47446ce86ecaf16fdde4fc4a60' into work_for_update_humble_tracepoint_added
…nt_added

chore: merge from humble 16.0.5
…ric subscription (tier4#7)

* added initialization tracepoint to generic_subscription

Signed-off-by: yamasaki <114902604+ymski@users.noreply.github.com>

* added callback_register

Signed-off-by: yamasaki <114902604+ymski@users.noreply.github.com>

* added callback_start/end

Signed-off-by: yamasaki <114902604+ymski@users.noreply.github.com>

* added subscription-side trace point

Signed-off-by: ymski <114902604+ymski@users.noreply.github.com>

* add rclcpp_publish to generic_publisher

Signed-off-by: ymski <114902604+ymski@users.noreply.github.com>

* removed unused dispatch_subscription_callback trace point

Signed-off-by: ymski <114902604+ymski@users.noreply.github.com>

* Update rclcpp/src/rclcpp/generic_subscription.cpp

Co-authored-by: isp-uetsuki <35490433+isp-uetsuki@users.noreply.github.com>

* fix messege addr

Signed-off-by: ymski <114902604+ymski@users.noreply.github.com>

* remove redundant includes

Signed-off-by: ymski <114902604+ymski@users.noreply.github.com>

---------

Signed-off-by: yamasaki <114902604+ymski@users.noreply.github.com>
Signed-off-by: ymski <114902604+ymski@users.noreply.github.com>
Co-authored-by: isp-uetsuki <35490433+isp-uetsuki@users.noreply.github.com>
…_container

Align ComponentManager with ros2/rclcpp humble (ros2#2363, ros2#2380):
set the QoS depth of load_node / unload_node services to 200.

Signed-off-by: nago-7563 <274183846+nago-7563@users.noreply.github.com>
@nago-7563
nago-7563 marked this pull request as draft October 29, 2026 11:10
@nago-7563 nago-7563 closed this Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants