You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Domain ID not changable on rclcpp::NodeOptions #910
rclcpp::NodeOptions has no way to set the Domain ID on a per-node basis, and it's kludgy to get the domain ID node_options->get_rcl_node_options()->domain_id. This functionality is exposed on rcl_node_options_t, so it's just a deficiency in the c++ API.
This makes sense to me, we'd appreciate anyone who has time to contribute this addition to the NodeOptions. We should consider doing it in rclpy at the same time, if it's not already there.
That's a good point @ivanpauno. In that case maybe we should hold off on this until we have a little more clear direction on that set of changes.
@suab321321 it sounds like we'd like to hold off on this for now. Our intention is to follow through with ros2/rmw#183, in which case this issue will no longer make sense. There's a chance we change our mind, in which case this issue would make sense again, but that is unlikely in my opinion.
I added a note to the issue description of ros2/rmw#183 that we should follow up on this issue once we've committed to doing that work or not.
kinda off topic from this thread, but the same goes to rclpy ?
actually rclpy does not have support for ROS_DOMAIN_ID (ros2/rclpy#484). i think that ROS_DOMAIN_ID should be supported via rcl and frontend(rclxxx) can ask the domain id to rcl. (just to reduce redundancy of implementation.)
actually rclpy does not have support for ROS_DOMAIN_ID (ros2/rclpy#484). i think that ROS_DOMAIN_ID should be supported via rcl and frontend(rclxxx) can ask the domain id to rcl. (just to reduce redundancy of implementation.)
The ROS_DOMAIN_ID logic is actually implemented in rcl, as you suggested.
If it's not working, it must be an error in how rclpy is wrapping rcl stuff.
and the question is why rclcpp has the same code to get the ROS_DOMAIN_ID?
It's needed just for better introspection. Currently, the default node options have a domain id value of SIZEMAX. When that value is detected in the node creation, it's updated to the value indicated by the env variable. To be able to introspect correctly the domain id if you call get_rcl_node_options before creating a node, the value is also get from the environment in that case.
i believe that we could delete the redundant code in rclcpp?
if so, i will make the issue and PR against it.
Yes, the redundant code can be avoided.
As part of ros2/rcl#515, I'm creating an rcl_domain_id function, that allows you to get the default domain_id from the environment.
Code can easily be reused after that gets merged.
i got a question, since Context is introduced and mapped to dds participant, I think that domain id also should be mapped to Context (same as security) not only for NodeOptions.
i am considering the following,
expose domain_id to NodeOptions
default std::numeric_limits<size_t>::max() and ROS_DOMAIN_ID variable is set via get_domain_id_from_env(), but user can overwrite the domain_id if needed. actually rmw_fastrtps/cyclonedds does not use this domain_id via rmw_node_create, so this is for implementation that can support domain_id with node level.
expose domain_id to InitOptions
this is actually effective with dds implementation such as fastrtps or cyclonedds. default std::numeric_limits<size_t>::max() and ROS_DOMAIN_ID variable is set, and user can overwrite the domain_id if needed.
We shouldn't expose nothing in NodeOptions, as that will not continue to be used.
Exposing it in InitOptions is ok, but I would prefer to have a cleaner API (i.e.: not exposing that size_t max means "default").
classInitOptions
{
...
voiduse_default_domain_id(); // this will take the domain id from the envvoidset_domain_id(size_t domain_id);
size_tget_domain_id() const;
...
}
I do agree with you, so maybe it is better to clean anything related to ROS_DOMAIN_ID stuff from NodeOptions. and we also update docs (if exist) explicitly about domain id with Context for user.
I do agree with you, so maybe it is better to clean anything related to ROS_DOMAIN_ID stuff from NodeOptions. and we also update docs (if exist) explicitly about domain id with Context for user.
If you want to clean that up, it's welcomed too! The only reference too domain_id in node related API is in rmw IIRC.
Exposing the domain id here is not related with that change, so it can be done separately.
Looks like the new API to support setting/getting domain IDs targeted post-Foxy distributions.
Does anyone involved in this thread know how to obtain the domain ID from a node in Foxy? The snippet below gives me a runtime error:
size_tget_domain_id_from_node(
const rclcpp::Node & node)
{
constrcl_init_options_t * rcl_init_options =
node.get_node_base_interface()->get_context()->get_init_options().get_rcl_init_options();
std::size_t domain_id;
// const_cast is safe because `rcl_init_options_get_domain_id` only reads the input structurercl_ret_t ret =
rcl_init_options_get_domain_id(
const_cast<rcl_init_options_t *>(rcl_init_options),
&domain_id);
if (RCL_RET_OK != ret) {
std::string err {"Failed to get domain ID from rcl_init_options: "};
err += std::to_string(domain_id);
throwstd::runtime_error(err);
}
}
$ ./domain_id_test
terminate called after throwing an instance of 'std::runtime_error'what(): Failed to get domain ID from rcl_init_options: 140276023001360
Aborted (core dumped)
@aprotyas you should check rcl_get_error_string() when an rcl function fails to see more info. The backtrace isn't useful because it's just showing us where you threw your own exception.
@aprotyas the latest version takes a pointer to const as the first argument:
@aprotyas you should check rcl_get_error_string() when an rcl function fails to see more info. The backtrace isn't useful because it's just showing us where you threw your own exception.
@wjwwood that's a good point, thanks! I figured out what happened from the rcl_get_error_string() and a Valgrind trace of my test program:
terminate called after throwing an instance of 'std::runtime_error'
what(): init_options->impl argument is null, at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/ros2/rcl/rcl/src/rcl/init_options.c:149
Aborted (core dumped)
==98403== Memcheck, a memory error detector
==98403== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==98403== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==98403== Command: ./build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id
==98403== Parent PID: 1
==98403==
==98403== Invalid read of size 8
==98403== at 0x505C2CA: rcl_init_options_get_domain_id (init_options.c:149)
==98403== by 0x110C4A: main (main.cpp:102)
==98403== Address 0x5b1bc20 is 0 bytes inside a block of size 8 free'd
==98403== at 0x483F1CF: operator delete(void*, unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==98403== by 0x4DDB25D: std::default_delete<rcl_init_options_t>::operator()(rcl_init_options_t*) const (unique_ptr.h:81)
==98403== by 0x4DDB13B: std::unique_ptr<rcl_init_options_t, std::default_delete<rcl_init_options_t> >::~unique_ptr() (unique_ptr.h:292)
==98403== by 0x4DDACD5: rclcpp::InitOptions::~InitOptions() (init_options.cpp:78)
==98403== by 0x110C22: main (main.cpp:98)
==98403== Block was alloc'd at
==98403== at 0x483DE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==98403== by 0x4DDA9E1: rclcpp::InitOptions::InitOptions(rcl_init_options_t const&) (init_options.cpp:34)
==98403== by 0x4DDAB19: rclcpp::InitOptions::InitOptions(rclcpp::InitOptions const&) (init_options.cpp:44)
==98403== by 0x4DA088C: rclcpp::Context::get_init_options() (context.cpp:275)
==98403== by 0x110BFD: main (main.cpp:98)
Since this constructor allocates new memory, when this line terminates, the rclcpp::InitOptions instance goes out of scope and the memory allocated for the struct rcl_init_options_t is free'd.
Replacing the problematic line with a block that allows me to get domain ID before the InitOptions instance goes out of scope works properly:
Activity
wjwwood commentedon Nov 15, 2019
This makes sense to me, we'd appreciate anyone who has time to contribute this addition to the NodeOptions. We should consider doing it in
rclpyat the same time, if it's not already there.suab321321 commentedon Nov 17, 2019
@wjwwood sir I would like to contribute
ivanpauno commentedon Nov 18, 2019
Note: When we finish with ros2/rmw#183, one domain id per node will have no more sense, and it will be one domain id per context.
wjwwood commentedon Nov 18, 2019
That's a good point @ivanpauno. In that case maybe we should hold off on this until we have a little more clear direction on that set of changes.
@suab321321 it sounds like we'd like to hold off on this for now. Our intention is to follow through with ros2/rmw#183, in which case this issue will no longer make sense. There's a chance we change our mind, in which case this issue would make sense again, but that is unlikely in my opinion.
I added a note to the issue description of ros2/rmw#183 that we should follow up on this issue once we've committed to doing that work or not.
suab321321 commentedon Nov 18, 2019
@wjwwood okay
fujitatomoya commentedon Dec 17, 2019
@wjwwood @ivanpauno
kinda off topic from this thread, but the same goes to rclpy ?
actually rclpy does not have support for ROS_DOMAIN_ID (ros2/rclpy#484). i think that ROS_DOMAIN_ID should be supported via rcl and frontend(rclxxx) can ask the domain id to rcl. (just to reduce redundancy of implementation.)
could you share your thoughts on this?
ivanpauno commentedon Dec 17, 2019
The
ROS_DOMAIN_IDlogic is actually implemented inrcl, as you suggested.If it's not working, it must be an error in how
rclpyis wrappingrclstuff.fujitatomoya commentedon Dec 18, 2019
@ivanpauno
sorry my bad, i should have checked.
https://github.com/ros2/rcl/blob/024c3ee50d9b79a85665efe1c191cc1225026519/rcl/src/rcl/node.c#L262-L281
and the question is why rclcpp has the same code to get the ROS_DOMAIN_ID?
rclcpp/rclcpp/src/rclcpp/node_options.cpp
Lines 325 to 355 in 0c66d0c
i believe that we could delete the redundant code in rclcpp?
if so, i will make the issue and PR against it.
ivanpauno commentedon Dec 18, 2019
It's needed just for better introspection. Currently, the default node options have a domain id value of
SIZEMAX. When that value is detected in the node creation, it's updated to the value indicated by the env variable. To be able to introspect correctly the domain id if you callget_rcl_node_optionsbefore creating a node, the value is also get from the environment in that case.Yes, the redundant code can be avoided.
As part of ros2/rcl#515, I'm creating an
rcl_domain_idfunction, that allows you to get the default domain_id from the environment.Code can easily be reused after that gets merged.
fujitatomoya commentedon Dec 19, 2019
@ivanpauno
got it, i will get it done after your PR is merged.
thanks
fujitatomoya commentedon Jun 4, 2020
@ivanpauno
i got a question, since Context is introduced and mapped to dds participant, I think that domain id also should be mapped to Context (same as security) not only for NodeOptions.
i am considering the following,
default
std::numeric_limits<size_t>::max()and ROS_DOMAIN_ID variable is set via get_domain_id_from_env(), but user can overwrite the domain_id if needed. actually rmw_fastrtps/cyclonedds does not use this domain_id via rmw_node_create, so this is for implementation that can support domain_id with node level.this is actually effective with dds implementation such as fastrtps or cyclonedds. default
std::numeric_limits<size_t>::max()and ROS_DOMAIN_ID variable is set, and user can overwrite the domain_id if needed.do you have any other thoughts?
ivanpauno commentedon Jun 4, 2020
We shouldn't expose nothing in
NodeOptions, as that will not continue to be used.Exposing it in
InitOptionsis ok, but I would prefer to have a cleaner API (i.e.: not exposing that size_t max means "default").fujitatomoya commentedon Jun 7, 2020
I do agree with you, so maybe it is better to clean anything related to ROS_DOMAIN_ID stuff from NodeOptions. and we also update docs (if exist) explicitly about domain id with Context for user.
fujitatomoya commentedon Jun 7, 2020
@suab321321
sorry i missed it, are you up for this?
ivanpauno commentedon Jun 8, 2020
If you want to clean that up, it's welcomed too! The only reference too
domain_idin node related API is inrmwIIRC.Exposing the domain id here is not related with that change, so it can be done separately.
fujitatomoya commentedon Jun 11, 2020
@ivanpauno
either me or someone else from our team can work on this, you can leave this on us. though, we would require your review for sure.
aprotyas commentedon May 28, 2022
Looks like the new API to support setting/getting domain IDs targeted post-Foxy distributions.
Does anyone involved in this thread know how to obtain the domain ID from a node in Foxy? The snippet below gives me a runtime error:
fujitatomoya commentedon May 31, 2022
ros2/rcl#946 backport foxy has been merged, so it should be working. can we take a look at the stack trace via gdb?
aprotyas commentedon May 31, 2022
@fujitatomoya I expected it to work too, but maybe you would have some insights from the backtrace:
Output of gdb exec -ex "run" -ex "thread apply all bt" -ex "quit"
Since it's pretty long, you can access it here too: https://pastebin.ubuntu.com/p/bFjpD3Rsn3/
wjwwood commentedon May 31, 2022
@aprotyas you should check
rcl_get_error_string()when an rcl function fails to see more info. The backtrace isn't useful because it's just showing us where you threw your own exception.@aprotyas the latest version takes a pointer to const as the first argument:
https://github.com/ros2/rcl/blob/72cee8a9fbda40304ff4f8e7c5d12b6ce6410f8d/rcl/include/rcl/init_options.h#L152
vs foxy:
https://github.com/ros2/rcl/blob/9b0b151a49ed9013c78520e738658c948bd93116/rcl/include/rcl/init_options.h#L150
Maybe there are other backports missing.
aprotyas commentedon May 31, 2022
@wjwwood that's a good point, thanks! I figured out what happened from the
rcl_get_error_string()and a Valgrind trace of my test program:The problem is this line:
get_init_options()invokes anInitOptionscopy constructor, which eventually calls:rclcpp/rclcpp/src/rclcpp/init_options.cpp
Lines 33 to 34 in 5c1dd19
Since this constructor allocates new memory, when this line terminates, the
rclcpp::InitOptionsinstance goes out of scope and the memory allocated for thestruct rcl_init_options_tis free'd.Replacing the problematic line with a block that allows me to get domain ID before the InitOptions instance goes out of scope works properly:
The error makes sense to me, but it was a bit unexpected. Should this behavior be expected?
Fix getting domain ID process
Expose the QoS object wrapper (ros2#910)