Skip to content

run_benchmarks script#27

Open
jordis-ai2 wants to merge 2 commits intomainfrom
eval_versions
Open

run_benchmarks script#27
jordis-ai2 wants to merge 2 commits intomainfrom
eval_versions

Conversation

@jordis-ai2
Copy link
Copy Markdown
Contributor

No description provided.

@jordis-ai2 jordis-ai2 marked this pull request as ready for review March 30, 2026 16:14
# Pick: 20s horizon → 303 steps @ 66ms policy_dt
# PnP: 40s horizon → 606 steps @ 66ms policy_dt
# ======================================================================
"pick_msproc": {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can these names be, to keep it more uniform

.
├── mb_pick_classic
├── mb_pick_filament
├── mb_pick_msproc
├── mb_pick_rand_cam
├── mb_pnp
├── mb_pnp_color
├── mb_pnp_next_to

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not directly run eval_to_csv.py using this script too and put the outputs in the folder format specified above.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The data below could almost be in a metadata and shipped togther witht he benchmarks. I guess there is not good place to put, or maybe we just call it molmobot_bench_metadata.json

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise this script will have to grow / be replicated for other benchmarks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants