Reliability issue?

[LanderU]

Hello!

I'm not sure if this is actually a problem with my set of test or with the code itself.

Description:

We're running long term tests using your tool.

performance_test commit: 04430ed

ROS_DISTRO: Dashing https://github.com/ros2/ros2/tree/release-dashing-20190614

Performance_test command:

ros2 run performance_test perf_test -c ROS2 -l log -t Array1k

We're running this in loopback.

Test result:

After more or less 4-5h, is not deterministic, the output stop get stuck and not more output is printed. We need this data in order to plot the results.

In order to check if this is on the computer that is running the tests. I run it on a official dashing docker, and the result is the same, in other machine.

GDB output:

(gdb) where full
#0  0x00005580183e7818 in performance_test::DataRunner<performance_test::ROS2CallbackCommunicator<performance_test::topics::Array1k> >::sync_reset()

We need infinitive tests for the application that we're building, I'm not sure if there is a problem in my setup or not.

It would be great if you can advise me. I can provide you more information if needed.

Thanks in advance!

[deeplearningrobotics]

@LanderU: So the tool is stuck in the sync_reset function? Do you know which line? Maybe this is a race condition.

[LanderU]

So the tool is stuck in the sync_reset function?

You're right.

These are the candidates:

• https://github.com/ApexAI/performance_test/blob/master/performance_test/src/experiment_execution/analyze_runner.cpp#L71
• https://github.com/ApexAI/performance_test/blob/master/performance_test/src/data_running/data_runner.hpp#L103

Thank you!

[deeplearningrobotics]

@LanderU: So it is stuck in the lock because another thread did not free it. Can you check for the other thread which is most likely stuck in the ROS 2 communicator class? The error you are seeing is just a symptom of this other thread being stuck.

[dejanpan]

@LanderU are you still having this problem or can we close this issue?

@esteve fyi

[LanderU]

The last time that I tested the problem still persists. I'll try it again.