-
Notifications
You must be signed in to change notification settings - Fork 72
Description
Summary
When using bondpy with lifecycle_manager, I encounter the following recurring error during node reactivation:
[save_path_action_server-12] [ERROR] [1749619669.362997625] [save_path_action_server]: More than two locations are trying to use a single bond (topic: bond, id: save_path_action_server). You should only instantiate at most two bond instances for each (topic, id) pair.
[multi_route_tracker-11] [ERROR] [1749619669.364058766] [multi_route_tracker]: More than two locations are trying to use a single bond (topic: bond, id: multi_route_tracker). You should only instantiate at most two bond instances for each (topic, id) pair.
[route_tracker-10] [ERROR] [1749619669.377074427] [route_tracker]: More than two locations are trying to use a single bond (topic: bond, id: route_tracker). You should only instantiate at most two bond instances for each (topic, id) pair.
[route_server-9] [ERROR] [1749619669.388306269] [route_server]: More than two locations are trying to use a single bond (topic: bond, id: route_server). You should only instantiate at most two bond instances for each (topic, id) pair.
[route_tracker-10] [ERROR] [1749619669.394350616] [route_tracker]: More than two locations are trying to use a single bond (topic: bond, id: route_tracker). You should only instantiate at most two bond instances for each (topic, id) pair.
This occurs when the node crashes or is deactivated, and the LifecycleManager automatically attempts to reconfigure and reactivate it. The previous bond instance (instance_id) is not properly cleaned up, which causes a conflict when the bond ID is reused.
Expected Behavior
Only one bond instance_id per bond id should remain active. When the node restarts (or transitions through deactivate → cleanup → configure → activate), the bond should clean up properly without triggering this error.
Actual Behavior
The bond is not broken when the node is stopped, deactivated, or crashes unexpectedly. On automatic reactivation by the lifecycle_manager, the same bond id is reused, and bondpy detects multiple active instance_ids, resulting in an error log spam.
Steps to Reproduce
Launch a lifecycle-managed node using bondpy with a static bond ID, e.g., save_path_action_server.
Let the node run and establish a bond.
Cause the node to crash or transition (e.g., by returning FAILURE from on_activate()).
LifecycleManager reactivates the node → a new bond instance is created.
Observe repeated log messages:
More than two locations are trying to use a single bond (topic: bond, id: save_path_action_server)
[save_path_action_server-12] [ERROR] [timestamp] [save_path_action_server]: More than two locations are trying to use a single bond (topic: bond, id: save_path_action_server).
[route_server-9] [ERROR] [timestamp] [route_server]: More than two locations are trying to use a single bond (topic: bond, id: route_server).
Environment
ROS 2 distro: Humble
bondpy: installed from binary
Node type: LifecycleNode
Managed by: lifecycle_manager or custom script
Launch method: [launch file, ros2 run, etc.]
Possible Cause
When the node exits or transitions without calling bond.break_bond(), the bond instance_id remains alive.
If LifecycleManager restarts the node quickly (before the bond's heartbeat_timeout expires), a second bond is created using the same id, causing the conflict.
Workarounds Tried
Manually added break_bond() in on_deactivate(), on_cleanup(), and on_shutdown()
Added signal handlers to break bond on SIGINT/SIGTERM
Added delay (sleep) after breaking bond
Used has_parameter() to avoid re-declaration errors
Yet the issue still occurs if the node crashes before cleanup can happen, and LifecycleManager tries to restart it.
What I'm Requesting
Guidance on how to reliably clean up bond state during crashes or fast restarts.
Suggestion on whether bondpy should auto-handle stale instance_ids more gracefully.
Or if LifecycleManager should delay reactivation to avoid race conditions with bond timeout.
If possible, expose cleanup hooks or bond management tools for lifecycle-managed nodes.