Skip to content

Organize benchmark test framework files under extended_tests#3427

Open
lajagapp wants to merge 2 commits intomainfrom
users/lajagapp/add-extended-tests
Open

Organize benchmark test framework files under extended_tests#3427
lajagapp wants to merge 2 commits intomainfrom
users/lajagapp/add-extended-tests

Conversation

@lajagapp
Copy link
Contributor

Motivation

This PR is the first part of restructuring the testing infrastructure. It focuses on file organization - moving existing benchmark test files from benchmarks/ to extended_tests/ to prepare for a unified test framework that will support both benchmark and functional tests.

Note: This PR is split from a larger change to make it easier to review and iterate. Functional test implementation will be added in a follow-up PR.

Technical Details

1. Directory Restructure

Moved: benchmarks/ to extended_tests/benchmark/

New structure:

extended_tests/
├── benchmark/          # Benchmark tests (moved from benchmarks/)
│   ├── scripts/       # Test implementations
│   ├── configs/       # Test configurations
│   └── ...
├── configs/           # Shared framework configuration
├── functional/        # Placeholder for functional tests (follow-up PR)
│   └── README.md     # Documentation placeholder
└── utils/             # Shared utilities (moved from benchmarks/utils/)

2. Key Changes

File Organization

  • Moved all files from benchmarks/ to extended_tests/benchmark/
  • Moved benchmarks/utils/ to extended_tests/utils/ (shared across test types)
  • Moved benchmarks/configs/config.yml to extended_tests/configs/config.yml (shared config)
  • Renamed benchmark_client.py to test_client.py (preparation for shared use)

Import Path Updates

  • Updated all Python imports in moved files to reflect new paths
  • Updated fetch_test_configurations.py
  • Updated configure_ci_test.py

Documentation

  • Created extended_tests/README.md: Framework overview with placeholder for functional tests
  • Created extended_tests/functional/README.md: Placeholder documentation (functional tests to be added in follow-up PR)
  • Updated all README files to reflect new directory structure and paths

Test Plan

Testing:

  • Existing benchmark tests should continue to work with updated import paths
  • CI should continue to function as before (no functional test workflow changes in this PR)

Next Steps

This PR establishes the file structure. Follow-up PRs will:

  1. Add functional test implementation (scripts, base class, test matrix)
  2. Add CI workflow integration for functional tests

Signed-off-by: Lenine Ajagappane <Lenine.Ajagappane@amd.com>
Signed-off-by: Lenine Ajagappane <Lenine.Ajagappane@amd.com>
@lajagapp lajagapp marked this pull request as ready for review February 16, 2026 18:43
Copy link
Contributor

@geomin12 geomin12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, can we add a test to make sure this works properly? a URL of a good run?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drive-by: I don't think these files belong under build_tools/github_actions/ at all. They aren't build tools and they shouldn't be unique to github actions.

Maybe a subfolder under tests/ is more appropriate?

Keeping files that aren't build tools in the build tools directory is making running unit tests unnecessarily complicated, see

  • # Note: build_tools/github_actions/ contains complicated tests in some
    # subdirectories that do not support pytest:
    # build_tools/github_actions/benchmarks/scripts/
    # build_tools/github_actions/test_executable_scripts/
    # We could move them if we want a simple `pytest build_tools/` to work.
    - name: Test build_tools
    working-directory: build_tools
    run: |
    python -m pytest -vv \
    --cov --cov-report=term-missing --cov-report=html
  • [tool.pytest.ini_options]
    testpaths = [
    "tests",
    "github_actions/tests",
    ]
    python_files = ["*_test.py"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: TODO

Development

Successfully merging this pull request may close these issues.

3 participants