ROS2 Humble Lifecycle Node Transition Command Segmentation fault (core dumped)

[isaiahoh]

Description

I get a Segmentation fault after the command when trying to run the ros2 lifecycle set <node_name> <state> commands. The results are below:

Environment

OS: Ubuntu 22.04
ROS2: Humble
Iceoryx: v2.0.3
rmw_iceoryx: v2.0.3-humble

Input:

RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 lifecycle set /ars548_driver configure

Output

Log level set to: [Warning]
Segmentation fault (core dumped)

Logs

iox-roudi

❯ iox-roudi -l verbose
2024-12-18 12:57:04.798 [ Info  ]: No config file provided and also not found at '/etc/iceoryx/roudi_config.toml'. Falling back to built-in config.
Log level set to: [Verbose]
2024-12-18 12:57:04.799 [Verbose]: Command line parameters are:
Log level: Verbose
Monitoring mode: MonitoringMode::ON
Compatibility check level: CompatibilityCheckLevel::PATCH
Unique RouDi ID: < unset >
Process kill delay: 45 s
Config file used is: < none >
Reserving 66761736 bytes in the shared memory [iceoryx_mgmt]
[ Reserving shared memory successful ]
2024-12-18 12:57:04.826 [ Debug ]: Registered memory segment 0x75cb58e54000 with size 66761736 to id 1
Reserving 149264720 bytes in the shared memory [isaiah]
[ Reserving shared memory successful ]
2024-12-18 12:57:04.872 [ Debug ]: Roudi registered payload data segment 0x75cb4fffa000 with size 149264720 to id 2
RouDi is ready for clients
2024-12-18 12:57:10.130 [ Debug ]: Registered new application python3_25379
2024-12-18 12:57:10.131 [ Debug ]: Created new SubscriberPort for application 'python3_25379' with service description 'Service: Introspection, Instance: RouDi_ID, Event: Port'
2024-12-18 12:57:10.131 [ Debug ]: Created new ConditionVariable for application python3_25379
2024-12-18 12:57:10.132 [ Debug ]: Created new node /_ros2cli_daemon_0_d767a1abb3f54e7386194416d153ac4b for process python3_25379
2024-12-18 12:57:10.132 [ Debug ]: Created new PublisherPort for application 'python3_25379' with service description 'Service: rcl_interfaces/msg/Log, Instance: /rosout, Event: data'
2024-12-18 12:57:10.146 [ Debug ]: Created new PublisherPort for application 'python3_25379' with service description 'Service: rcl_interfaces/msg/ParameterEvent, Instance: /parameter_events, Event: data'
2024-12-18 12:57:10.146 [ Debug ]: Created new ConditionVariable for application python3_25379
2024-12-18 12:57:17.257 [ Debug ]: Registered new application ars548_driver_25438
2024-12-18 12:57:17.258 [ Debug ]: Created new SubscriberPort for application 'ars548_driver_25438' with service description 'Service: Introspection, Instance: RouDi_ID, Event: Port'
2024-12-18 12:57:17.258 [ Debug ]: Created new ConditionVariable for application ars548_driver_25438
2024-12-18 12:57:17.258 [ Debug ]: Created new node /ars548_driver for process ars548_driver_25438
2024-12-18 12:57:17.258 [ Debug ]: Created new PublisherPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/msg/Log, Instance: /rosout, Event: data'
2024-12-18 12:57:17.258 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/srv/GetParameters, Instance: /ars548_driver/get_parameters, Event: data'
2024-12-18 12:57:17.259 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/srv/GetParameterTypes, Instance: /ars548_driver/get_parameter_types, Event: data'
2024-12-18 12:57:17.259 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/srv/SetParameters, Instance: /ars548_driver/set_parameters, Event: data'
2024-12-18 12:57:17.259 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/srv/SetParametersAtomically, Instance: /ars548_driver/set_parameters_atomically, Event: data'
2024-12-18 12:57:17.259 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/srv/DescribeParameters, Instance: /ars548_driver/describe_parameters, Event: data'
2024-12-18 12:57:17.259 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/srv/ListParameters, Instance: /ars548_driver/list_parameters, Event: data'
2024-12-18 12:57:17.259 [ Debug ]: Created new PublisherPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/msg/ParameterEvent, Instance: /parameter_events, Event: data'
2024-12-18 12:57:17.259 [ Debug ]: Created new SubscriberPort for application 'ars548_driver_25438' with service description 'Service: rcl_interfaces/msg/ParameterEvent, Instance: /parameter_events, Event: data'
2024-12-18 12:57:17.260 [ Debug ]: Created new PublisherPort for application 'ars548_driver_25438' with service description 'Service: lifecycle_msgs/msg/TransitionEvent, Instance: /ars548_driver/transition_event, Event: data'
2024-12-18 12:57:17.260 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: lifecycle_msgs/srv/ChangeState, Instance: /ars548_driver/change_state, Event: data'
2024-12-18 12:57:17.260 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: lifecycle_msgs/srv/GetState, Instance: /ars548_driver/get_state, Event: data'
2024-12-18 12:57:17.260 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: lifecycle_msgs/srv/GetAvailableStates, Instance: /ars548_driver/get_available_states, Event: data'
2024-12-18 12:57:17.260 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: lifecycle_msgs/srv/GetAvailableTransitions, Instance: /ars548_driver/get_available_transitions, Event: data'
2024-12-18 12:57:17.260 [ Debug ]: Created new ServerPort for application 'ars548_driver_25438' with service description 'Service: lifecycle_msgs/srv/GetAvailableTransitions, Instance: /ars548_driver/get_transition_graph, Event: data'
2024-12-18 12:57:17.261 [ Debug ]: Created new ConditionVariable for application ars548_driver_25438
2024-12-18 12:57:28.641 [ Debug ]: Created new SubscriberPort for application 'python3_25379' with service description 'Service: Introspection, Instance: RouDi_ID, Event: Process'
2024-12-18 12:57:28.642 [ Debug ]: Created new SubscriberPort for application 'python3_25379' with service description 'Service: ServiceDiscovery, Instance: RouDi_ID, Event: ServiceRegistry'
2024-12-18 12:57:28.643 [ Debug ]: Registered new application python3_25471
2024-12-18 12:57:28.644 [ Debug ]: Created new SubscriberPort for application 'python3_25471' with service description 'Service: Introspection, Instance: RouDi_ID, Event: Port'
2024-12-18 12:57:28.644 [ Debug ]: Created new ConditionVariable for application python3_25471
2024-12-18 12:57:28.644 [ Debug ]: Created new node /_ros2cli_25471 for process python3_25471
2024-12-18 12:57:28.645 [ Debug ]: Created new PublisherPort for application 'python3_25471' with service description 'Service: rcl_interfaces/msg/Log, Instance: /rosout, Event: data'
2024-12-18 12:57:28.659 [ Debug ]: Created new PublisherPort for application 'python3_25471' with service description 'Service: rcl_interfaces/msg/ParameterEvent, Instance: /parameter_events, Event: data'
2024-12-18 12:57:28.659 [ Debug ]: Created new ConditionVariable for application python3_25471
2024-12-18 12:57:28.722 [ Debug ]: Destroy subscriber port from runtime 'python3_25379' and with service description 'Service: ServiceDiscovery, Instance: RouDi_ID, Event: ServiceRegistry'
2024-12-18 12:57:29.169 [ Debug ]: Created new ClientPort for application 'python3_25471' with service description 'Service: lifecycle_msgs/srv/GetAvailableTransitions, Instance: /ars548_driver/get_available_transitions, Event: data'
2024-12-18 12:57:30.170 [ Debug ]: Created new ConditionVariable for application python3_25471
2024-12-18 12:57:30.225 [ Debug ]: Destroy ConditionVariableData from runtime 'python3_25471'
2024-12-18 12:57:31.729 [Warning]: Application python3_25471 not responding (last response 1583 milliseconds ago) --> removing it
2024-12-18 12:57:31.729 [ Debug ]: Destroy publisher port from runtime 'python3_25471' and with service description 'Service: rcl_interfaces/msg/Log, Instance: /rosout, Event: data'
2024-12-18 12:57:31.729 [ Debug ]: Destroy publisher port from runtime 'python3_25471' and with service description 'Service: rcl_interfaces/msg/ParameterEvent, Instance: /parameter_events, Event: data'
2024-12-18 12:57:31.729 [ Debug ]: Destroy subscriber port from runtime 'python3_25471' and with service description 'Service: Introspection, Instance: RouDi_ID, Event: Port'
2024-12-18 12:57:31.729 [ Debug ]: Destroy client port from runtime 'python3_25471' and with service description 'Service: lifecycle_msgs/srv/GetAvailableTransitions, Instance: /ars548_driver/get_available_transitions, Event: data'
2024-12-18 12:57:31.729 [ Debug ]: Deleted node of application python3_25471
2024-12-18 12:57:31.729 [ Debug ]: Deleted condition variable of applicationpython3_25471
2024-12-18 12:57:31.729 [ Debug ]: Deleted condition variable of applicationpython3_25471

valgrind

valgrind ros2 lifecycle set /ars548_driver configure
==26852== Memcheck, a memory error detector
==26852== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==26852== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==26852== Command: /opt/ros/humble/bin/ros2 lifecycle set /ars548_driver configure
==26852==
Log level set to: [Warning]
==26852== Invalid write of size 1
==26852==    at 0x8443D42: char const* rmw_iceoryx_cpp::details_c::deserialize_message_field<unsigned char>(rosidl_typesupport_introspection_c__MessageMember_s const*, char const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x8442EB7: rmw_iceoryx_cpp::details_c::deserialize(char const*, rosidl_typesupport_introspection_c__MessageMembers_s const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x8443187: rmw_iceoryx_cpp::details_c::deserialize(char const*, rosidl_typesupport_introspection_c__MessageMembers_s const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x84430AB: rmw_iceoryx_cpp::details_c::deserialize(char const*, rosidl_typesupport_introspection_c__MessageMembers_s const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x8423C8E: iox::cxx::function_ref<void (void const*&)>::function_ref<rmw_take_response::{lambda(void const*)#1}, void>(rmw_take_response::{lambda(void const*)#1}&&)::{lambda(void*, void const*&)#1}::_FUN(void*, void const*&) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_cpp.so)
==26852==    by 0x842418D: rmw_take_response (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_cpp.so)
==26852==    by 0x802176C: rcl_take_response_with_info (in /opt/ros/humble/lib/librcl.so)
==26852==    by 0x798EA78: ??? (in /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so)
==26852==    by 0x798C395: ??? (in /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so)
==26852==    by 0x796B1B8: ??? (in /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so)
==26852==    by 0x293281: ??? (in /usr/bin/python3.10)
==26852==    by 0x289B4A: _PyObject_MakeTpCall (in /usr/bin/python3.10)
==26852==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==26852==
==26852==
==26852== Process terminating with default action of signal 11 (SIGSEGV)
==26852==  Access not within mapped region at address 0x0
==26852==    at 0x8443D42: char const* rmw_iceoryx_cpp::details_c::deserialize_message_field<unsigned char>(rosidl_typesupport_introspection_c__MessageMember_s const*, char const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x8442EB7: rmw_iceoryx_cpp::details_c::deserialize(char const*, rosidl_typesupport_introspection_c__MessageMembers_s const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x8443187: rmw_iceoryx_cpp::details_c::deserialize(char const*, rosidl_typesupport_introspection_c__MessageMembers_s const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x84430AB: rmw_iceoryx_cpp::details_c::deserialize(char const*, rosidl_typesupport_introspection_c__MessageMembers_s const*, void*) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_serialization.so)
==26852==    by 0x8423C8E: iox::cxx::function_ref<void (void const*&)>::function_ref<rmw_take_response::{lambda(void const*)#1}, void>(rmw_take_response::{lambda(void const*)#1}&&)::{lambda(void*, void const*&)#1}::_FUN(void*, void const*&) (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_cpp.so)
==26852==    by 0x842418D: rmw_take_response (in /ros_ws/build/rmw_iceoryx_cpp/librmw_iceoryx_cpp.so)
==26852==    by 0x802176C: rcl_take_response_with_info (in /opt/ros/humble/lib/librcl.so)
==26852==    by 0x798EA78: ??? (in /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so)
==26852==    by 0x798C395: ??? (in /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so)
==26852==    by 0x796B1B8: ??? (in /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so)
==26852==    by 0x293281: ??? (in /usr/bin/python3.10)
==26852==    by 0x289B4A: _PyObject_MakeTpCall (in /usr/bin/python3.10)
==26852==  If you believe this happened as a result of a stack
==26852==  overflow in your program's main thread (unlikely but
==26852==  possible), you can try to increase the size of the
==26852==  main thread stack using the --main-stacksize= flag.
==26852==  The main thread stack size used in this run was 8388608.
==26852==
==26852== HEAP SUMMARY:
==26852==     in use at exit: 9,822,516 bytes in 12,053 blocks
==26852==   total heap usage: 52,364 allocs, 40,311 frees, 101,946,825 bytes allocated
==26852==
==26852== LEAK SUMMARY:
==26852==    definitely lost: 0 bytes in 0 blocks
==26852==    indirectly lost: 0 bytes in 0 blocks
==26852==      possibly lost: 247,593 bytes in 230 blocks
==26852==    still reachable: 9,574,923 bytes in 11,823 blocks
==26852==         suppressed: 0 bytes in 0 blocks
==26852== Rerun with --leak-check=full to see details of leaked memory
==26852==
==26852== For lists of detected and suppressed errors, rerun with: -s
==26852== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

[mossmaurice]

@isaiahoh Thanks for reporting this issue!

I tried to reproduce the issue today with the following steps:

# Created ADE ros2_humble environment based on public image: https://gitlab.com/ApexAI/ros2-ade
mkdir -p ~/ade-home/ros2_humble
cd ~/ade-home/ros2_humble
get https://gitlab.com/ApexAI/ros2-ade/-/raw/master/.aderc-humble -O .aderc
ade start --update --enter

# Build steps inside ADE/Docker
source /opt/ros/humble/setup.bash
cd ~/ros2_humble
mkdir src
wget https://raw.githubusercontent.com/ros2/ros2/humble/ros2.repos
vcs import src < ros2.repos
cd ~/ros2_humble/src/
git clone --branch humble https://github.com/ros2/rmw_iceoryx.git
cd ~/ros2_humble
colcon build --symlink-install --merge-install

## Run the Lifecycle demo from https://github.com/ros2/demos/blob/humble/lifecycle/README.rst

# Terminal 1
source install/setup.bash
iox-roudi -l verbose -m off

# Terminal 2
source install/setup.bash
RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 run lifecycle lifecycle_talker

# Terminal 3
source install/setup.bash
RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 run lifecycle lifecycle_service_client

# Terminal 4
source install/setup.bash
RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 run lifecycle lifecycle_listener

The lifecycle demo ran fine without errors.

Could you please provide a reproducer for this issue i.e. push your example onto a branch? Thanks!

[isaiahoh]

@mossmaurice
Here is a repo to reproduce the issue, with instructions and more information on the error in the README.md
https://github.com/isaiahoh/ros2_humble_iceoryx
Let me know if you require any info that is missing! Merry Christmas!

[mossmaurice]

@isaiahoh My apologies for the late reply! Thank you for the reproducer and the instructions 🚀 I was able to reproduce the segfault and will have a look.

[EdisonAltamirano]

any update on this ?