Skip to content

Conversation

@connoraird
Copy link
Contributor

Description

Use mean instead of min with a more lenient threshold and use different timer to exclude IO and sleep

Brought over from glass-benchmarks repo PR glass-dev/glass-benchmarks#20

Closes: #784

Checks

  • Is your code passing linting?
  • Is your code passing tests?
  • Have you added additional tests (if required)?
  • Have you modified/extended the documentation (if required)?
  • Have you added a one-liner changelog entry above (if required)?

@connoraird connoraird self-assigned this Nov 13, 2025
@connoraird connoraird added the benchmarks Benchmarking work label Nov 13, 2025
@connoraird connoraird changed the title Use mean instead of min with a more lenient threshold and use differe… gh-21: Improve reg test reliability Nov 13, 2025
Copy link
Member

@paddyroddy paddyroddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10% seems better and hopefully this will prove stable

@connoraird connoraird merged commit ab28ea3 into main Nov 13, 2025
16 of 18 checks passed
@connoraird connoraird deleted the connor/issue-21 branch November 13, 2025 17:05
@paddyroddy paddyroddy changed the title gh-21: Improve reg test reliability gh-784: Improve reg test reliability Dec 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmarks Benchmarking work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve reliability of regression tests

3 participants