Skip to content

Domain ID not changable on rclcpp::NodeOptions #910

Closed

Description

@rotu

rclcpp::NodeOptions has no way to set the Domain ID on a per-node basis, and it's kludgy to get the domain ID node_options->get_rcl_node_options()->domain_id. This functionality is exposed on rcl_node_options_t, so it's just a deficiency in the c++ API.

Activity

  1. self-assigned this
    on Nov 14, 2019
  2. wjwwood commented on Nov 15, 2019

    @wjwwood
    Member

    This makes sense to me, we'd appreciate anyone who has time to contribute this addition to the NodeOptions. We should consider doing it in rclpy at the same time, if it's not already there.

  3. suab321321 commented on Nov 17, 2019

    @suab321321
    Contributor

    @wjwwood sir I would like to contribute

  4. ivanpauno commented on Nov 18, 2019

    @ivanpauno
    Member

    Note: When we finish with ros2/rmw#183, one domain id per node will have no more sense, and it will be one domain id per context.

  5. wjwwood commented on Nov 18, 2019

    @wjwwood
    Member

    That's a good point @ivanpauno. In that case maybe we should hold off on this until we have a little more clear direction on that set of changes.

    @suab321321 it sounds like we'd like to hold off on this for now. Our intention is to follow through with ros2/rmw#183, in which case this issue will no longer make sense. There's a chance we change our mind, in which case this issue would make sense again, but that is unlikely in my opinion.

    I added a note to the issue description of ros2/rmw#183 that we should follow up on this issue once we've committed to doing that work or not.

  6. suab321321 commented on Nov 18, 2019

    @suab321321
    Contributor
  7. fujitatomoya commented on Dec 17, 2019

    @fujitatomoya
    Collaborator

    @wjwwood @ivanpauno

    kinda off topic from this thread, but the same goes to rclpy ?

    actually rclpy does not have support for ROS_DOMAIN_ID (ros2/rclpy#484). i think that ROS_DOMAIN_ID should be supported via rcl and frontend(rclxxx) can ask the domain id to rcl. (just to reduce redundancy of implementation.)

    could you share your thoughts on this?

  8. ivanpauno commented on Dec 17, 2019

    @ivanpauno
    Member

    actually rclpy does not have support for ROS_DOMAIN_ID (ros2/rclpy#484). i think that ROS_DOMAIN_ID should be supported via rcl and frontend(rclxxx) can ask the domain id to rcl. (just to reduce redundancy of implementation.)

    The ROS_DOMAIN_ID logic is actually implemented in rcl, as you suggested.
    If it's not working, it must be an error in how rclpy is wrapping rcl stuff.

  9. fujitatomoya commented on Dec 18, 2019

    @fujitatomoya
    Collaborator

    @ivanpauno

    The ROS_DOMAIN_ID logic is actually implemented in rcl, as you suggested.

    sorry my bad, i should have checked.

    https://github.com/ros2/rcl/blob/024c3ee50d9b79a85665efe1c191cc1225026519/rcl/src/rcl/node.c#L262-L281

    and the question is why rclcpp has the same code to get the ROS_DOMAIN_ID?

    // TODO(wjwwood): reuse rcutils_get_env() to avoid code duplication.
    // See also: https://github.com/ros2/rcl/issues/119
    size_t
    NodeOptions::get_domain_id_from_env() const
    {
    // Determine the domain id based on the options and the ROS_DOMAIN_ID env variable.
    size_t domain_id = std::numeric_limits<size_t>::max();
    char * ros_domain_id = nullptr;
    const char * env_var = "ROS_DOMAIN_ID";
    #ifndef _WIN32
    ros_domain_id = getenv(env_var);
    #else
    size_t ros_domain_id_size;
    _dupenv_s(&ros_domain_id, &ros_domain_id_size, env_var);
    #endif
    if (ros_domain_id) {
    uint32_t number = strtoul(ros_domain_id, NULL, 0);
    if (number == (std::numeric_limits<uint32_t>::max)()) {
    #ifdef _WIN32
    // free the ros_domain_id before throwing, if getenv was used on Windows
    free(ros_domain_id);
    #endif
    throw std::runtime_error("failed to interpret ROS_DOMAIN_ID as integral number");
    }
    domain_id = static_cast<size_t>(number);
    #ifdef _WIN32
    free(ros_domain_id);
    #endif
    }
    return domain_id;
    }

    i believe that we could delete the redundant code in rclcpp?
    if so, i will make the issue and PR against it.

  10. ivanpauno commented on Dec 18, 2019

    @ivanpauno
    Member

    and the question is why rclcpp has the same code to get the ROS_DOMAIN_ID?

    It's needed just for better introspection. Currently, the default node options have a domain id value of SIZEMAX. When that value is detected in the node creation, it's updated to the value indicated by the env variable. To be able to introspect correctly the domain id if you call get_rcl_node_options before creating a node, the value is also get from the environment in that case.

    i believe that we could delete the redundant code in rclcpp?
    if so, i will make the issue and PR against it.

    Yes, the redundant code can be avoided.
    As part of ros2/rcl#515, I'm creating an rcl_domain_id function, that allows you to get the default domain_id from the environment.
    Code can easily be reused after that gets merged.

  11. fujitatomoya commented on Dec 19, 2019

    @fujitatomoya
    Collaborator

    @ivanpauno

    got it, i will get it done after your PR is merged.

    thanks

  12. fujitatomoya commented on Jun 4, 2020

    @fujitatomoya
    Collaborator

    @ivanpauno

    i got a question, since Context is introduced and mapped to dds participant, I think that domain id also should be mapped to Context (same as security) not only for NodeOptions.

    i am considering the following,

    • expose domain_id to NodeOptions
      default std::numeric_limits<size_t>::max() and ROS_DOMAIN_ID variable is set via get_domain_id_from_env(), but user can overwrite the domain_id if needed. actually rmw_fastrtps/cyclonedds does not use this domain_id via rmw_node_create, so this is for implementation that can support domain_id with node level.
    • expose domain_id to InitOptions
      this is actually effective with dds implementation such as fastrtps or cyclonedds. default std::numeric_limits<size_t>::max() and ROS_DOMAIN_ID variable is set, and user can overwrite the domain_id if needed.

    do you have any other thoughts?

  13. ivanpauno commented on Jun 4, 2020

    @ivanpauno
    Member

    We shouldn't expose nothing in NodeOptions, as that will not continue to be used.
    Exposing it in InitOptions is ok, but I would prefer to have a cleaner API (i.e.: not exposing that size_t max means "default").

    class InitOptions
    {
    ...
    void use_default_domain_id();  // this will take the domain id from the env
    void set_domain_id(size_t domain_id); 
    size_t get_domain_id() const; 
    ...
    }
  14. fujitatomoya commented on Jun 7, 2020

    @fujitatomoya
    Collaborator

    I do agree with you, so maybe it is better to clean anything related to ROS_DOMAIN_ID stuff from NodeOptions. and we also update docs (if exist) explicitly about domain id with Context for user.

  15. fujitatomoya commented on Jun 7, 2020

    @fujitatomoya
    Collaborator

    @suab321321

    sorry i missed it, are you up for this?

  16. ivanpauno commented on Jun 8, 2020

    @ivanpauno
    Member

    I do agree with you, so maybe it is better to clean anything related to ROS_DOMAIN_ID stuff from NodeOptions. and we also update docs (if exist) explicitly about domain id with Context for user.

    If you want to clean that up, it's welcomed too! The only reference too domain_id in node related API is in rmw IIRC.
    Exposing the domain id here is not related with that change, so it can be done separately.

  17. fujitatomoya commented on Jun 11, 2020

    @fujitatomoya
    Collaborator

    @ivanpauno

    either me or someone else from our team can work on this, you can leave this on us. though, we would require your review for sure.

  18. aprotyas commented on May 28, 2022

    @aprotyas
    Member

    Looks like the new API to support setting/getting domain IDs targeted post-Foxy distributions.

    Does anyone involved in this thread know how to obtain the domain ID from a node in Foxy? The snippet below gives me a runtime error:

    size_t
    get_domain_id_from_node(
      const rclcpp::Node & node)
    {
      const rcl_init_options_t * rcl_init_options = 
        node.get_node_base_interface()->get_context()->get_init_options().get_rcl_init_options();
    
      std::size_t domain_id;
      // const_cast is safe because `rcl_init_options_get_domain_id` only reads the input structure
      rcl_ret_t ret =
        rcl_init_options_get_domain_id(
        const_cast<rcl_init_options_t *>(rcl_init_options),
        &domain_id);
      if (RCL_RET_OK != ret) {
        std::string err {"Failed to get domain ID from rcl_init_options: "};
        err += std::to_string(domain_id);
        throw std::runtime_error(err);
      }
    }
    $ ./domain_id_test
    terminate called after throwing an instance of 'std::runtime_error'
      what():  Failed to get domain ID from rcl_init_options: 140276023001360
    Aborted (core dumped)
  19. fujitatomoya commented on May 31, 2022

    @fujitatomoya
    Collaborator

    ros2/rcl#946 backport foxy has been merged, so it should be working. can we take a look at the stack trace via gdb?

  20. aprotyas commented on May 31, 2022

    @aprotyas
    Member

    @fujitatomoya I expected it to work too, but maybe you would have some insights from the backtrace:

    Output of gdb exec -ex "run" -ex "thread apply all bt" -ex "quit"
    $ gdb build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id -ex "run" -ex "thread apply all bt" -ex "quit"
    
    GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
    Copyright (C) 2020 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Type "show copying" and "show warranty" for details.
    This GDB was configured as "x86_64-linux-gnu".
    Type "show configuration" for configuration details.
    For bug reporting instructions, please see:
    <http://www.gnu.org/software/gdb/bugs/>.
    Find the GDB manual and other documentation resources online at:
        <http://www.gnu.org/software/gdb/documentation/>.
    
    For help, type "help".
    Type "apropos word" to search for commands related to "word"...
    Reading symbols from build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id...
    Starting program: /home/aprotyas/dev/workspaces/ros2_playground_ws/build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id 
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
    [New Thread 0x7ffff5723700 (LWP 97916)]
    [New Thread 0x7ffff4e99700 (LWP 97917)]
    [New Thread 0x7ffff4698700 (LWP 97918)]
    [New Thread 0x7ffff3e97700 (LWP 97919)]
    [New Thread 0x7ffff3689700 (LWP 97920)]
    [New Thread 0x7ffff2e88700 (LWP 97921)]
    [New Thread 0x7ffff267a700 (LWP 97922)]
    [New Thread 0x7ffff1d79700 (LWP 97923)]
    [New Thread 0x7ffff1565700 (LWP 97924)]
    
    Thread 1 "rclcpp_foxy_dom" received signal SIGABRT, Aborted.
    __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
    
    Thread 10 (Thread 0x7ffff1565700 (LWP 97924)):
    #0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7fffe4000ba8) at ../sysdeps/nptl/futex-internal.h:183
    #1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7fffe4000bb0, cond=0x7fffe4000b80) at pthread_cond_wait.c:508
    #2  __pthread_cond_wait (cond=0x7fffe4000b80, mutex=0x7fffe4000bb0) at pthread_cond_wait.c:647
    #3  0x00007ffff7658e30 in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #4  0x00007ffff6f7567d in std::condition_variable::wait<rmw_fastrtps_shared_cpp::__rmw_wait(char const*, rmw_subscriptions_t*, rmw_guard_conditions_t*, rmw_services_t*, rmw_clients_t*, rmw_events_t*, rmw_wait_set_t*, const rmw_time_t*)::<lambda()> >(std::unique_lock<std::mutex> &, rmw_fastrtps_shared_cpp::<lambda()>) (this=0x7fffe4000b80, __lock=..., __p=...) at /usr/include/c++/9/condition_variable:101
    #5  0x00007ffff6f75153 in rmw_fastrtps_shared_cpp::__rmw_wait (identifier=0x7ffff701bd00 "rmw_fastrtps_cpp", subscriptions=0x7ffff1564560, guard_conditions=0x7ffff1564570, services=0x0, clients=0x0, events=0x0, wait_set=0x7fffe4000b60, wait_timeout=0x0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/rmw_wait.cpp:173
    #6  0x00007ffff6f33296 in node_listener (context=0x5555555da350) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/listener_thread.cpp:136
    #7  0x00007ffff6f34385 in std::__invoke_impl<void, void (*)(rmw_context_t*), rmw_context_t*> (__f=@0x5555556ba8e0: 0x7ffff6f33075 <node_listener(rmw_context_t*)>) at /usr/include/c++/9/bits/invoke.h:60
    #8  0x00007ffff6f342d1 in std::__invoke<void (*)(rmw_context_t*), rmw_context_t*> (__fn=@0x5555556ba8e0: 0x7ffff6f33075 <node_listener(rmw_context_t*)>) at /usr/include/c++/9/bits/invoke.h:95
    #9  0x00007ffff6f34221 in std::thread::_Invoker<std::tuple<void (*)(rmw_context_t*), rmw_context_t*> >::_M_invoke<0ul, 1ul> (this=0x5555556ba8d8) at /usr/include/c++/9/thread:244
    #10 0x00007ffff6f341c3 in std::thread::_Invoker<std::tuple<void (*)(rmw_context_t*), rmw_context_t*> >::operator() (this=0x5555556ba8d8) at /usr/include/c++/9/thread:251
    #11 0x00007ffff6f34194 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(rmw_context_t*), rmw_context_t*> > >::_M_run (this=0x5555556ba8d0) at /usr/include/c++/9/thread:195
    #12 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #13 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #14 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 9 (Thread 0x7ffff1d79700 (LWP 97923)):
    #0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x5555555dcd4c) at ../sysdeps/nptl/futex-internal.h:183
    #1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5555555dbc00, cond=0x5555555dcd20) at pthread_cond_wait.c:508
    #2  __pthread_cond_wait (cond=0x5555555dcd20, mutex=0x5555555dbc00) at pthread_cond_wait.c:647
    #3  0x00007ffff7658e30 in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #4  0x00007ffff64646c5 in std::_V2::condition_variable_any::wait<std::unique_lock<std::recursive_timed_mutex> > (this=0x5555555dcd20, __lock=...) at /usr/include/c++/9/condition_variable:273
    #5  0x00007ffff64623d4 in eprosima::fastrtps::rtps::AsyncWriterThread::run (this=0x5555555dcc80) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/resources/AsyncWriterThread.cpp:135
    #6  0x00007ffff646501f in std::__invoke_impl<void, void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> (__f=@0x555555623d80: (void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(class eprosima::fastrtps::rtps::AsyncWriterThread * const)) 0x7ffff64622c0 <eprosima::fastrtps::rtps::AsyncWriterThread::run()>, __t=@0x555555623d78: 0x5555555dcc80) at /usr/include/c++/9/bits/invoke.h:73
    #7  0x00007ffff6464f39 in std::__invoke<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> (__fn=@0x555555623d80: (void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(class eprosima::fastrtps::rtps::AsyncWriterThread * const)) 0x7ffff64622c0 <eprosima::fastrtps::rtps::AsyncWriterThread::run()>) at /usr/include/c++/9/bits/invoke.h:95
    #8  0x00007ffff6464e89 in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> >::_M_invoke<0ul, 1ul> (this=0x555555623d78) at /usr/include/c++/9/thread:244
    #9  0x00007ffff6464e2b in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> >::operator() (this=0x555555623d78) at /usr/include/c++/9/thread:251
    #10 0x00007ffff6464dfc in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> > >::_M_run (this=0x555555623d70) at /usr/include/c++/9/thread:195
    #11 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #12 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #13 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 8 (Thread 0x7ffff267a700 (LWP 97922)):
    #0  0x00007ffff736a678 in futex_abstimed_wait_cancelable (private=128, abstime=0x7ffff2678d20, clockid=0, expected=0, futex_word=0x7ffff2680110) at ../sysdeps/nptl/futex-internal.h:320
    #1  do_futex_wait (sem=sem@entry=0x7ffff2680110, abstime=abstime@entry=0x7ffff2678d20, clockid=0) at sem_waitcommon.c:112
    #2  0x00007ffff736a7a3 in __new_sem_wait_slow (sem=0x7ffff2680110, abstime=0x7ffff2678d20, clockid=0) at sem_waitcommon.c:184
    #3  0x00007ffff6920707 in boost::interprocess::ipcdetail::semaphore_timed_wait (handle=0x7ffff2680110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore_wrapper.hpp:226
    #4  0x00007ffff6920c5d in boost::interprocess::ipcdetail::posix_semaphore::timed_wait (this=0x7ffff2680110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore.hpp:55
    #5  0x00007ffff6920ccf in boost::interprocess::interprocess_semaphore::timed_wait (this=0x7ffff2680110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/interprocess_semaphore.hpp:139
    #6  0x00007ffff692143e in eprosima::fastdds::rtps::RobustInterprocessCondition::do_timed_wait (this=0x7ffff267b138, abs_time=..., mut=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:347
    #7  0x00007ffff692fd91 in eprosima::fastdds::rtps::RobustInterprocessCondition::timed_wait<std::unique_lock<boost::interprocess::interprocess_mutex>, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}>(std::unique_lock<boost::interprocess::interprocess_mutex>&, boost::posix_time::ptime const&, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}) (this=0x7ffff267b138, lock=..., abs_time=..., pred=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:148
    #8  0x00007ffff6924fdd in eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop (this=0x555555618d80, listener=..., is_listener_closed=..., listener_index=0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemGlobal.hpp:549
    #9  0x00007ffff692945c in eprosima::fastdds::rtps::SharedMemManager::Listener::pop (this=0x555555618b10) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemManager.hpp:695
    #10 0x00007ffff692cdb0 in eprosima::fastdds::rtps::SharedMemChannelResource::Receive (this=0x555555618e50, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:182
    #11 0x00007ffff692c928 in eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation (this=0x555555618e50, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:133
    #12 0x00007ffff6992a38 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x55555561b818: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x55555561b810: 0x555555618e50) at /usr/include/c++/9/bits/invoke.h:73
    #13 0x00007ffff6992503 in std::__invoke<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x55555561b818: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
    #14 0x00007ffff699229f in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x55555561b7f8) at /usr/include/c++/9/thread:244
    #15 0x00007ffff6991c25 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x55555561b7f8) at /usr/include/c++/9/thread:251
    #16 0x00007ffff6990fd6 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x55555561b7f0) at /usr/include/c++/9/thread:195
    #17 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #18 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #19 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 7 (Thread 0x7ffff2e88700 (LWP 97921)):
    #0  __libc_recvmsg (flags=0, msg=0x7ffff2e86f60, fd=10) at ../sysdeps/unix/sysv/linux/recvmsg.c:28
    #1  __libc_recvmsg (fd=10, msg=0x7ffff2e86f60, flags=0) at ../sysdeps/unix/sysv/linux/recvmsg.c:25
    #2  0x00007ffff65f70cc in asio::detail::socket_ops::recvfrom (s=10, bufs=0x7ffff2e870a0, count=1, flags=0, addr=0x7ffff2e871d0, addrlen=0x7ffff2e87090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:939
    #3  0x00007ffff65f71c5 in asio::detail::socket_ops::sync_recvfrom (s=10, state=32 ' ', bufs=0x7ffff2e870a0, count=1, flags=0, addr=0x7ffff2e871d0, addrlen=0x7ffff2e87090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:961
    #4  0x00007ffff65f8bd7 in asio::detail::reactive_socket_service<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555dc0e0, impl=..., buffers=..., sender_endpoint=..., flags=0, ec=...) at /usr/include/asio/detail/reactive_socket_service.hpp:298
    #5  0x00007ffff65f8593 in asio::basic_datagram_socket<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555f5150, buffers=..., sender_endpoint=...) at /usr/include/asio/basic_datagram_socket.hpp:858
    #6  0x00007ffff65f5491 in eprosima::fastdds::rtps::UDPChannelResource::Receive (this=0x5555555f5110, receive_buffer=0x5555556085e0 "", receive_buffer_capacity=65500, receive_buffer_size=@0x5555555f512c: 0, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:90
    #7  0x00007ffff65f522f in eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation (this=0x5555555f5110, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:61
    #8  0x00007ffff65f95d5 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555f53d8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555f53d0: 0x5555555f5110) at /usr/include/c++/9/bits/invoke.h:73
    #9  0x00007ffff65f9481 in std::__invoke<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555f53d8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
    #10 0x00007ffff65f9391 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555f53b8) at /usr/include/c++/9/thread:244
    #11 0x00007ffff65f9317 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555f53b8) at /usr/include/c++/9/thread:251
    #12 0x00007ffff65f92e8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555f53b0) at /usr/include/c++/9/thread:195
    #13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 6 (Thread 0x7ffff3689700 (LWP 97920)):
    #0  0x00007ffff736a678 in futex_abstimed_wait_cancelable (private=128, abstime=0x7ffff3687d20, clockid=0, expected=0, futex_word=0x7ffff368f110) at ../sysdeps/nptl/futex-internal.h:320
    #1  do_futex_wait (sem=sem@entry=0x7ffff368f110, abstime=abstime@entry=0x7ffff3687d20, clockid=0) at sem_waitcommon.c:112
    #2  0x00007ffff736a7a3 in __new_sem_wait_slow (sem=0x7ffff368f110, abstime=0x7ffff3687d20, clockid=0) at sem_waitcommon.c:184
    #3  0x00007ffff6920707 in boost::interprocess::ipcdetail::semaphore_timed_wait (handle=0x7ffff368f110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore_wrapper.hpp:226
    #4  0x00007ffff6920c5d in boost::interprocess::ipcdetail::posix_semaphore::timed_wait (this=0x7ffff368f110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore.hpp:55
    #5  0x00007ffff6920ccf in boost::interprocess::interprocess_semaphore::timed_wait (this=0x7ffff368f110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/interprocess_semaphore.hpp:139
    #6  0x00007ffff692143e in eprosima::fastdds::rtps::RobustInterprocessCondition::do_timed_wait (this=0x7ffff368a138, abs_time=..., mut=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:347
    #7  0x00007ffff692fd91 in eprosima::fastdds::rtps::RobustInterprocessCondition::timed_wait<std::unique_lock<boost::interprocess::interprocess_mutex>, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}>(std::unique_lock<boost::interprocess::interprocess_mutex>&, boost::posix_time::ptime const&, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}) (this=0x7ffff368a138, lock=..., abs_time=..., pred=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:148
    #8  0x00007ffff6924fdd in eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop (this=0x5555555f5260, listener=..., is_listener_closed=..., listener_index=0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemGlobal.hpp:549
    #9  0x00007ffff692945c in eprosima::fastdds::rtps::SharedMemManager::Listener::pop (this=0x5555555f5030) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemManager.hpp:695
    #10 0x00007ffff692cdb0 in eprosima::fastdds::rtps::SharedMemChannelResource::Receive (this=0x5555555f53f0, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:182
    #11 0x00007ffff692c928 in eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation (this=0x5555555f53f0, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:133
    #12 0x00007ffff6992a38 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555f54a8: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555f54a0: 0x5555555f53f0) at /usr/include/c++/9/bits/invoke.h:73
    #13 0x00007ffff6992503 in std::__invoke<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555f54a8: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
    #14 0x00007ffff699229f in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555f5488) at /usr/include/c++/9/thread:244
    #15 0x00007ffff6991c25 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555f5488) at /usr/include/c++/9/thread:251
    #16 0x00007ffff6990fd6 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555f5480) at /usr/include/c++/9/thread:195
    #17 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #18 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #19 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 5 (Thread 0x7ffff3e97700 (LWP 97919)):
    #0  __libc_recvmsg (flags=0, msg=0x7ffff3e95f60, fd=8) at ../sysdeps/unix/sysv/linux/recvmsg.c:28
    #1  __libc_recvmsg (fd=8, msg=0x7ffff3e95f60, flags=0) at ../sysdeps/unix/sysv/linux/recvmsg.c:25
    #2  0x00007ffff65f70cc in asio::detail::socket_ops::recvfrom (s=8, bufs=0x7ffff3e960a0, count=1, flags=0, addr=0x7ffff3e961d0, addrlen=0x7ffff3e96090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:939
    #3  0x00007ffff65f71c5 in asio::detail::socket_ops::sync_recvfrom (s=8, state=32 ' ', bufs=0x7ffff3e960a0, count=1, flags=0, addr=0x7ffff3e961d0, addrlen=0x7ffff3e96090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:961
    #4  0x00007ffff65f8bd7 in asio::detail::reactive_socket_service<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555dc0e0, impl=..., buffers=..., sender_endpoint=..., flags=0, ec=...) at /usr/include/asio/detail/reactive_socket_service.hpp:298
    #5  0x00007ffff65f8593 in asio::basic_datagram_socket<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555f4b20, buffers=..., sender_endpoint=...) at /usr/include/asio/basic_datagram_socket.hpp:858
    #6  0x00007ffff65f5491 in eprosima::fastdds::rtps::UDPChannelResource::Receive (this=0x5555555f4ae0, receive_buffer=0x5555555f58a0 "", receive_buffer_capacity=65500, receive_buffer_size=@0x5555555f4afc: 0, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:90
    #7  0x00007ffff65f522f in eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation (this=0x5555555f4ae0, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:61
    #8  0x00007ffff65f95d5 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555e41b8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555e41b0: 0x5555555f4ae0) at /usr/include/c++/9/bits/invoke.h:73
    #9  0x00007ffff65f9481 in std::__invoke<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555e41b8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
    #10 0x00007ffff65f9391 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555e4198) at /usr/include/c++/9/thread:244
    #11 0x00007ffff65f9317 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555e4198) at /usr/include/c++/9/thread:251
    #12 0x00007ffff65f92e8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555e4190) at /usr/include/c++/9/thread:195
    #13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 4 (Thread 0x7ffff4698700 (LWP 97918)):
    #0  __libc_recvmsg (flags=0, msg=0x7ffff4696f60, fd=7) at ../sysdeps/unix/sysv/linux/recvmsg.c:28
    #1  __libc_recvmsg (fd=7, msg=0x7ffff4696f60, flags=0) at ../sysdeps/unix/sysv/linux/recvmsg.c:25
    #2  0x00007ffff65f70cc in asio::detail::socket_ops::recvfrom (s=7, bufs=0x7ffff46970a0, count=1, flags=0, addr=0x7ffff46971d0, addrlen=0x7ffff4697090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:939
    #3  0x00007ffff65f71c5 in asio::detail::socket_ops::sync_recvfrom (s=7, state=32 ' ', bufs=0x7ffff46970a0, count=1, flags=0, addr=0x7ffff46971d0, addrlen=0x7ffff4697090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:961
    #4  0x00007ffff65f8bd7 in asio::detail::reactive_socket_service<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555dc0e0, impl=..., buffers=..., sender_endpoint=..., flags=0, ec=...) at /usr/include/asio/detail/reactive_socket_service.hpp:298
    #5  0x00007ffff65f8593 in asio::basic_datagram_socket<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555e3fd0, buffers=..., sender_endpoint=...) at /usr/include/asio/basic_datagram_socket.hpp:858
    #6  0x00007ffff65f5491 in eprosima::fastdds::rtps::UDPChannelResource::Receive (this=0x5555555e3f90, receive_buffer=0x5555555e4a40 "RTPS\002\003\001\017\001\017\353}x~Jl\001", receive_buffer_capacity=65500, receive_buffer_size=@0x5555555e3fac: 264, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:90
    #7  0x00007ffff65f522f in eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation (this=0x5555555e3f90, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:61
    #8  0x00007ffff65f95d5 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555dc228: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555dc220: 0x5555555e3f90) at /usr/include/c++/9/bits/invoke.h:73
    #9  0x00007ffff65f9481 in std::__invoke<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555dc228: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
    #10 0x00007ffff65f9391 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555dc208) at /usr/include/c++/9/thread:244
    #11 0x00007ffff65f9317 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555dc208) at /usr/include/c++/9/thread:251
    #12 0x00007ffff65f92e8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555dc200) at /usr/include/c++/9/thread:195
    #13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 3 (Thread 0x7ffff4e99700 (LWP 97917)):
    #0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7ffff4e98470, clockid=<optimized out>, expected=0, futex_word=0x5555555dcb88) at ../sysdeps/nptl/futex-internal.h:320
    #1  __pthread_cond_wait_common (abstime=0x7ffff4e98470, clockid=<optimized out>, mutex=0x5555555dc380, cond=0x5555555dcb60) at pthread_cond_wait.c:520
    #2  __pthread_cond_timedwait (cond=0x5555555dcb60, mutex=0x5555555dc380, abstime=0x7ffff4e98470) at pthread_cond_wait.c:665
    #3  0x00007ffff7d23c8f in __gthread_cond_timedwait (__cond=0x5555555dcb60, __mutex=0x5555555dc380, __abs_timeout=0x7ffff4e98470) at /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:872
    #4  0x00007ffff7d29bd4 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dcb60, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:188
    #5  0x00007ffff7d27d90 in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dcb60, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:121
    #6  0x00007ffff645bb0b in std::_V2::condition_variable_any::wait_until<std::unique_lock<std::timed_mutex>, std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dcb60, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:296
    #7  0x00007ffff645a236 in eprosima::fastrtps::rtps::ResourceEvent::event_service (this=0x5555555dcae8) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/resources/ResourceEvent.cpp:182
    #8  0x00007ffff646077b in std::__invoke_impl<void, void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> (__f=@0x5555555df430: (void (eprosima::fastrtps::rtps::ResourceEvent::*)(class eprosima::fastrtps::rtps::ResourceEvent * const)) 0x7ffff645a0e0 <eprosima::fastrtps::rtps::ResourceEvent::event_service()>, __t=@0x5555555df428: 0x5555555dcae8) at /usr/include/c++/9/bits/invoke.h:73
    #9  0x00007ffff6460695 in std::__invoke<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> (__fn=@0x5555555df430: (void (eprosima::fastrtps::rtps::ResourceEvent::*)(class eprosima::fastrtps::rtps::ResourceEvent * const)) 0x7ffff645a0e0 <eprosima::fastrtps::rtps::ResourceEvent::event_service()>) at /usr/include/c++/9/bits/invoke.h:95
    #10 0x00007ffff64605e5 in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> >::_M_invoke<0ul, 1ul> (this=0x5555555df428) at /usr/include/c++/9/thread:244
    #11 0x00007ffff6460587 in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> >::operator() (this=0x5555555df428) at /usr/include/c++/9/thread:251
    #12 0x00007ffff6460558 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> > >::_M_run (this=0x5555555df420) at /usr/include/c++/9/thread:195
    #13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 2 (Thread 0x7ffff5723700 (LWP 97916)):
    #0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7ffff5722470, clockid=<optimized out>, expected=0, futex_word=0x5555555dc330) at ../sysdeps/nptl/futex-internal.h:320
    #1  __pthread_cond_wait_common (abstime=0x7ffff5722470, clockid=<optimized out>, mutex=0x5555555dc338, cond=0x5555555dc308) at pthread_cond_wait.c:520
    #2  __pthread_cond_timedwait (cond=0x5555555dc308, mutex=0x5555555dc338, abstime=0x7ffff5722470) at pthread_cond_wait.c:665
    #3  0x00007ffff7d23c8f in __gthread_cond_timedwait (__cond=0x5555555dc308, __mutex=0x5555555dc338, __abs_timeout=0x7ffff5722470) at /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:872
    #4  0x00007ffff7d29bd4 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dc308, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:188
    #5  0x00007ffff7d27d90 in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dc308, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:121
    #6  0x00007ffff6934d8c in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> >, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}>(std::unique_lock<std::mutex>&, std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > const&, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}) (this=0x5555555dc308, __lock=..., __atime=..., __p=...) at /usr/include/c++/9/condition_variable:138
    #7  0x00007ffff692f0eb in std::condition_variable::wait_for<long, std::ratio<1l, 1000l>, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}>(std::unique_lock<std::mutex>&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}) (this=0x5555555dc308, __lock=..., __rtime=..., __p=...) at /usr/include/c++/9/condition_variable:166
    #8  0x00007ffff69236c4 in eprosima::fastdds::rtps::SharedMemWatchdog::run (this=0x5555555dc2a0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemWatchdog.hpp:128
    #9  0x00007ffff6992b20 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> (__f=@0x5555555dee80: (void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(class eprosima::fastdds::rtps::SharedMemWatchdog * const)) 0x7ffff692364a <eprosima::fastdds::rtps::SharedMemWatchdog::run()>, __t=@0x5555555dee78: 0x5555555dc2a0) at /usr/include/c++/9/bits/invoke.h:73
    #10 0x00007ffff69925c9 in std::__invoke<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> (__fn=@0x5555555dee80: (void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(class eprosima::fastdds::rtps::SharedMemWatchdog * const)) 0x7ffff692364a <eprosima::fastdds::rtps::SharedMemWatchdog::run()>) at /usr/include/c++/9/bits/invoke.h:95
    #11 0x00007ffff69922f1 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> >::_M_invoke<0ul, 1ul> (this=0x5555555dee78) at /usr/include/c++/9/thread:244
    #12 0x00007ffff6991c67 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> >::operator() (this=0x5555555dee78) at /usr/include/c++/9/thread:251
    #13 0x00007ffff6990ffa in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> > >::_M_run (this=0x5555555dee70) at /usr/include/c++/9/thread:195
    #14 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #15 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
    #16 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    Thread 1 (Thread 0x7ffff704ef40 (LWP 97912)):
    #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
    #1  0x00007ffff739d859 in __GI_abort () at abort.c:79
    #2  0x00007ffff7626911 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #3  0x00007ffff763238c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #4  0x00007ffff76323f7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #5  0x00007ffff76326a9 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
    #6  0x000055555555cda1 in get_domain_id_from_node (node=std::shared_ptr<class rclcpp::Node> (use count 1, weak count 1) = {...}) at /home/aprotyas/dev/workspaces/ros2_playground_ws/src/ros2_playground/rclcpp_foxy_domain_id/src/main.cpp:61
    #7  0x000055555555ceef in main () at /home/aprotyas/dev/workspaces/ros2_playground_ws/src/ros2_playground/rclcpp_foxy_domain_id/src/main.cpp:70
    A debugging session is active.
    
    	Inferior 1 [process 97912] will be killed.
    
    Quit anyway? (y or n) y

    Since it's pretty long, you can access it here too: https://pastebin.ubuntu.com/p/bFjpD3Rsn3/

  21. wjwwood commented on May 31, 2022

    @wjwwood
    Member

    @aprotyas you should check rcl_get_error_string() when an rcl function fails to see more info. The backtrace isn't useful because it's just showing us where you threw your own exception.

    @aprotyas the latest version takes a pointer to const as the first argument:

    https://github.com/ros2/rcl/blob/72cee8a9fbda40304ff4f8e7c5d12b6ce6410f8d/rcl/include/rcl/init_options.h#L152

    vs foxy:

    https://github.com/ros2/rcl/blob/9b0b151a49ed9013c78520e738658c948bd93116/rcl/include/rcl/init_options.h#L150

    Maybe there are other backports missing.

  22. aprotyas commented on May 31, 2022

    @aprotyas
    Member

    @aprotyas you should check rcl_get_error_string() when an rcl function fails to see more info. The backtrace isn't useful because it's just showing us where you threw your own exception.

    @wjwwood that's a good point, thanks! I figured out what happened from the rcl_get_error_string() and a Valgrind trace of my test program:

    terminate called after throwing an instance of 'std::runtime_error'
      what():  init_options->impl argument is null, at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/ros2/rcl/rcl/src/rcl/init_options.c:149
    Aborted (core dumped)
    
    ==98403== Memcheck, a memory error detector
    ==98403== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
    ==98403== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
    ==98403== Command: ./build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id
    ==98403== Parent PID: 1
    ==98403== 
    ==98403== Invalid read of size 8
    ==98403==    at 0x505C2CA: rcl_init_options_get_domain_id (init_options.c:149)
    ==98403==    by 0x110C4A: main (main.cpp:102)
    ==98403==  Address 0x5b1bc20 is 0 bytes inside a block of size 8 free'd
    ==98403==    at 0x483F1CF: operator delete(void*, unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
    ==98403==    by 0x4DDB25D: std::default_delete<rcl_init_options_t>::operator()(rcl_init_options_t*) const (unique_ptr.h:81)
    ==98403==    by 0x4DDB13B: std::unique_ptr<rcl_init_options_t, std::default_delete<rcl_init_options_t> >::~unique_ptr() (unique_ptr.h:292)
    ==98403==    by 0x4DDACD5: rclcpp::InitOptions::~InitOptions() (init_options.cpp:78)
    ==98403==    by 0x110C22: main (main.cpp:98)
    ==98403==  Block was alloc'd at
    ==98403==    at 0x483DE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
    ==98403==    by 0x4DDA9E1: rclcpp::InitOptions::InitOptions(rcl_init_options_t const&) (init_options.cpp:34)
    ==98403==    by 0x4DDAB19: rclcpp::InitOptions::InitOptions(rclcpp::InitOptions const&) (init_options.cpp:44)
    ==98403==    by 0x4DA088C: rclcpp::Context::get_init_options() (context.cpp:275)
    ==98403==    by 0x110BFD: main (main.cpp:98)
    

    The problem is this line:

    const rcl_init_options_t * rcl_init_options = 
        node.get_node_base_interface()->get_context()->get_init_options().get_rcl_init_options();

    get_init_options() invokes an InitOptions copy constructor, which eventually calls:

    InitOptions::InitOptions(const rcl_init_options_t & init_options)
    : init_options_(new rcl_init_options_t)

    Since this constructor allocates new memory, when this line terminates, the rclcpp::InitOptions instance goes out of scope and the memory allocated for the struct rcl_init_options_t is free'd.

    Replacing the problematic line with a block that allows me to get domain ID before the InitOptions instance goes out of scope works properly:

    const auto init_options = node->get_node_options().context()->get_init_options();                                                                                               
    const rcl_init_options_t * rcl_init_options_ptr = __init_options.get_rcl_init_options();
    
    // get_domain_id  

    The error makes sense to me, but it was a bit unexpected. Should this behavior be expected?

  23. added a commit that references this issue on Jun 1, 2022
  24. added a commit that references this issue on Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions