Skip to content

Conversation

@azazdeaz
Copy link
Member

Main changes:

  • The waypoint test generates a metrics.json with the completion time for each waypoint session. This would be useful to compare policies.
  • Bail test early when the robot is stuck. This makes the tests suits with failing policies a lot faster because it doesn't have to wait for the timeout with an idling robot.
  • Fix dora-rs error when tests a are failing. dora-rs threw an error in the and if tests failed because the pytest process returned with a non-zero exit code
image

https://app.artefacts.com/artefacts-demos/go2-demo/job/775eed02-0657-4836-9b96-efc81697f450

@azazdeaz azazdeaz requested review from fabid and tomonorman December 22, 2025 15:56


@pytest.fixture(scope="session")
def metrics():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually thats a good point. Given we now have better support for usign metrics.json maybe we need a toolkit helper to actually help make / populate it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could be nice! Right now, i only add metrics from the test file, but mainly because it would need a more complex util to update metrics safely from multiple processes. It would be great if the toolkit could help with that.

import msgs


class StuckDetector:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think this is relatively generic? (toolkit?)

Copy link
Member Author

@azazdeaz azazdeaz Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about this, but probably every use-case is different enough so you cant use the same thing unless we make it complex with a lot of parameters. Might be better to have recepie-book or something with testing tricks like this.

Copy link
Contributor

@tomonorman tomonorman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@azazdeaz azazdeaz merged commit af3d694 into main Dec 23, 2025
1 check passed
@azazdeaz azazdeaz deleted the metrics-time branch December 23, 2025 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants