Benchmark comparison for rigid-body dynamic simulators. This project is part of GSoC'24@OpenRobotics.

This project is an effort to create an open-source benchmarking suite for robotics physics engines/simulators (for example: gazebo, mujoco, and drake).

Gazebo Classic benchmark is available here

• Boxes benchmark: Free-floating rigid bodies:

  

• Triball benchmark: Rigid bodies in contact:

pip install lz4 protobuf zstandard matplotlib numpy pandas
sudo apt-get install python3-gz-math7

• Gazebo Ionic:

  Installation

  • Follow the Gazebo Ionic source installation up to here.
  • Edit collection-ionic.yaml.
  • In gz-sim version: main, replace main with scepters/set_model_state_prototype.

  This is how it should look:

  gz-sim:
  type: git
  url: https://github.com/gazebosim/gz-sim
  version: scpeters/set_model_state_prototype

  Follow rest of the installation tutorial.

git clone https://github.com/yaswanth1701/simulation_benchmark.git
cd benchmark
git submodule update --init --recursive
mkdir build
cd build
cmake ..
make 
make test 

• boxes_results.ipynb

• Simulation & logging -> mcap to csv log conversion -> post_processing -> result_plots

• Logging (mcap & csv formats)
• Dynamics world generation at run-time(sdf.erb format).
• Simulator independent post-processing script.

These features allow for a simulator-independent suite for benchmarking.

• Gazebo Ionic benchmark:

  • DART
  • Bullet
  • Bullet-Featherstone

• Benchmarks with the model having joints and multi-body dynamics.
• Integration of Mujoco and Drake.

We would highly appreciate developers of open-source simulators and the robotics community suggesting or implementing new benchmarking ideas using this benchmarking suite and opening a pull request for it. If anyone comes across problems while running the benchmark, please let us know by opening an issue.