Skip to content

Added support for error based metrics#3

Merged
Thematiq merged 5 commits intomasterfrom
feat/add_regression_support
Feb 23, 2026
Merged

Added support for error based metrics#3
Thematiq merged 5 commits intomasterfrom
feat/add_regression_support

Conversation

@Thematiq
Copy link
Copy Markdown
Member

No description provided.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for error-based metrics (like RMSE) where lower scores indicate better performance, complementing the existing support for metrics where higher scores are better (like accuracy or F1). The implementation introduces a maximize parameter that, when set to False, internally negates the scores before computing win/tie/loss comparisons.

Changes:

  • Added maximize boolean parameter to PyBBT class with default value of True
  • Modified _construct_win_table to negate scores when maximize=False
  • Added test cases verifying correct behavior for both maximize modes
  • Updated README documentation with usage example for error-based metrics

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
bbttest/bbt/py_bbt.py Added maximize parameter to PyBBT class constructor and docstring; parameter is passed to _construct_win_table during fitting
bbttest/bbt/alg.py Modified _construct_win_table to accept maximize parameter and negate algorithm scores when maximize=False; added warning for unnamed columns; changed from set to list for algorithm names
tests/bbt/test_alg.py Added parametrized test cases for _construct_win_table with maximize=True and maximize=False; added test for unnamed column warning
README.md Added new section documenting how to use the maximize parameter for error-based metrics

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Thematiq Thematiq merged commit c17a0a3 into master Feb 23, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants