[AIROCMLIR-43] tuningRunner improvements - Add state file for crash recovery and improve logging#2208
Merged
mirza-halilcevic merged 40 commits intodevelopfrom Feb 20, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR enhances the tuning infrastructure with state file management for crash recovery and comprehensive logging improvements. The changes enable the tuner to track configuration states (running, failed, crashed, interrupted), persist them across runs, and recover gracefully from interruptions or crashes.
Changes:
- Added JSON state file mechanism to track tuning progress and enable crash recovery
- Introduced structured logging with color-coded output and tqdm integration
- Enhanced error reporting with detailed context and formatted output
- Added
--retry-failedflag to selectively retry failed/crashed configs - Improved progress tracking with ETA estimation based on median completion times
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| mlir/utils/performance/tuningRunner.py | Core implementation of state management, logging infrastructure, and enhanced error handling |
| mlir/utils/performance/perfRunner.py | Simplified tuning database reader to handle variable column counts |
| mlir/utils/jenkins/Jenkinsfile.downstream | Removed --quiet flag from CI tuning commands |
| mlir/utils/jenkins/Jenkinsfile | Removed --quiet flag from fusion tuning commands |
| mlir/lib/Dialect/Rock/Tuning/RockTuningImpl.cpp | Removed obsolete comment about hidden warning |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mirza-halilcevic
requested review from
dhernandez0,
dorde-antic,
pabloantoniom and
umangyadav
mirza-halilcevic
changed the title
1 task
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
apwojcik
pushed a commit
that referenced
this pull request
Feb 21, 2026
…ecovery and improve logging (#2208) * Improve time and metadata tracking, output file format has changed. * Add support for stdin. * Revert to old output format. * Add state file for crash and interrupt recovery. * Add --retry-failed option. * Use proper python logger. * Improve readability of output. * Log warnings from tuning driver. * Reintroduce --quiet flag. * Let important exceptions propagate and clean up code. * Simplify state file and support multiple contexts. * Address copilot comments. * Show tuning-driver output during failures. * Add --status option. * Improve order of logs for easier tracking. * Update github ci python version. * Fix state transitions. * Add timeout option. * Improve output file format. * Simplify output file writing. * Address code review comments. * Use llvm dbgs instead of errs where appropriate. * Warn if env vars are set.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Improve crash recovery, informational output, and error reporting.
Technical Details
Test Plan
This branch was used to create the tuning databases from which the quick-tune lists in #2212 were generated.
Submission Checklist