-
Notifications
You must be signed in to change notification settings - Fork 0
Improve Scripts and Tools Structure #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
f6d81a9
refactor(scripts,tools): consolidate coverage and lint tools infrastr…
tembo[bot] 82186f7
Update scripts/coverage/runner.py
JustAGhosT 62c1b6b
Update tools/core/cli/base.py
JustAGhosT bc06bd8
fix(cli): update path processing to handle non-existent paths and imp…
tembo[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,43 +1,12 @@ | ||
| # Check test coverage and enforce quality gates | ||
| # This script delegates to the unified Python coverage utility | ||
|
|
||
| param( | ||
| [int]$CoverageThreshold = 70 | ||
| ) | ||
|
|
||
| $ErrorActionPreference = 'Stop' | ||
|
|
||
| $CoverageFile = "coverage.xml" | ||
|
|
||
| Write-Host "========================================" -ForegroundColor Cyan | ||
| Write-Host "CodeFlow Engine - Coverage Check" -ForegroundColor Cyan | ||
| Write-Host "========================================" -ForegroundColor Cyan | ||
| Write-Host "" | ||
|
|
||
| # Check if coverage file exists | ||
| if (-not (Test-Path $CoverageFile)) { | ||
| Write-Host "⚠️ Coverage file not found. Running tests with coverage..." -ForegroundColor Yellow | ||
| poetry run pytest --cov=codeflow_engine --cov-report=xml --cov-report=term | ||
| } | ||
|
|
||
| # Get current coverage percentage | ||
| $CoverageOutput = poetry run coverage report | Select-String "TOTAL" | ||
| $Coverage = [regex]::Match($CoverageOutput, '(\d+(?:\.\d+)?)%').Groups[1].Value | ||
| $CoverageValue = [double]$Coverage | ||
|
|
||
| Write-Host "Current coverage: ${Coverage}%" | ||
| Write-Host "Target coverage: ${CoverageThreshold}%" | ||
| Write-Host "" | ||
|
|
||
| # Check if coverage meets threshold | ||
| if ($CoverageValue -lt $CoverageThreshold) { | ||
| Write-Host "❌ Coverage ${Coverage}% is below threshold of ${CoverageThreshold}%" -ForegroundColor Red | ||
| Write-Host "" | ||
| Write-Host "Coverage by module:" -ForegroundColor Yellow | ||
| poetry run coverage report --show-missing | Select-String "codeflow_engine" | Select-Object -First 20 | ||
| exit 1 | ||
| } | ||
| else { | ||
| Write-Host "✅ Coverage ${Coverage}% meets threshold of ${CoverageThreshold}%" -ForegroundColor Green | ||
| exit 0 | ||
| } | ||
| # Use the unified Python coverage utility | ||
| python -m scripts.coverage.runner check --threshold $CoverageThreshold | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,38 +1,11 @@ | ||
| #!/bin/bash | ||
| # Check test coverage and enforce quality gates | ||
| # This script delegates to the unified Python coverage utility | ||
|
|
||
| set -e | ||
|
|
||
| COVERAGE_THRESHOLD=${1:-70} | ||
| COVERAGE_FILE="coverage.xml" | ||
|
|
||
| echo "==========================================" | ||
| echo "CodeFlow Engine - Coverage Check" | ||
| echo "==========================================" | ||
| echo "" | ||
|
|
||
| # Check if coverage file exists | ||
| if [ ! -f "$COVERAGE_FILE" ]; then | ||
| echo "⚠️ Coverage file not found. Running tests with coverage..." | ||
| poetry run pytest --cov=codeflow_engine --cov-report=xml --cov-report=term | ||
| fi | ||
|
|
||
| # Get current coverage percentage | ||
| COVERAGE=$(poetry run coverage report | grep TOTAL | awk '{print $NF}' | sed 's/%//') | ||
|
|
||
| echo "Current coverage: ${COVERAGE}%" | ||
| echo "Target coverage: ${COVERAGE_THRESHOLD}%" | ||
| echo "" | ||
|
|
||
| # Check if coverage meets threshold | ||
| if (( $(echo "$COVERAGE < $COVERAGE_THRESHOLD" | bc -l) )); then | ||
| echo "❌ Coverage ${COVERAGE}% is below threshold of ${COVERAGE_THRESHOLD}%" | ||
| echo "" | ||
| echo "Coverage by module:" | ||
| poetry run coverage report --show-missing | grep -E "codeflow_engine" | head -20 | ||
| exit 1 | ||
| else | ||
| echo "✅ Coverage ${COVERAGE}% meets threshold of ${COVERAGE_THRESHOLD}%" | ||
| exit 0 | ||
| fi | ||
| # Use the unified Python coverage utility | ||
| python -m scripts.coverage.runner check --threshold "$COVERAGE_THRESHOLD" | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| """Coverage utilities for test quality gates.""" | ||
|
|
||
| from scripts.coverage.runner import CoverageRunner | ||
|
|
||
| __all__ = ["CoverageRunner"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,297 @@ | ||
| """Unified coverage runner for test quality gates. | ||
|
|
||
| This module provides a cross-platform Python implementation for running | ||
| test coverage, replacing the duplicated bash and PowerShell scripts. | ||
| """ | ||
|
|
||
| import argparse | ||
| import re | ||
| import subprocess | ||
| import sys | ||
| from dataclasses import dataclass | ||
| from pathlib import Path | ||
|
|
||
|
|
||
| @dataclass | ||
| class CoverageResult: | ||
| """Result of a coverage run.""" | ||
|
|
||
| total_coverage: float | ||
| by_module: dict[str, float] | ||
| low_coverage_files: list[tuple[str, float]] | ||
| no_coverage_files: list[str] | ||
| success: bool | ||
| error_message: str = "" | ||
|
|
||
|
|
||
| class CoverageRunner: | ||
| """Unified coverage runner for running tests and generating reports.""" | ||
|
|
||
| def __init__( | ||
| self, | ||
| module_name: str = "codeflow_engine", | ||
| threshold: float = 70.0, | ||
| low_coverage_threshold: float = 50.0, | ||
| ): | ||
| """Initialize the coverage runner. | ||
|
|
||
| Args: | ||
| module_name: The Python module to measure coverage for | ||
| threshold: Minimum coverage percentage required to pass | ||
| low_coverage_threshold: Threshold below which files are flagged | ||
| """ | ||
| self.module_name = module_name | ||
| self.threshold = threshold | ||
| self.low_coverage_threshold = low_coverage_threshold | ||
|
|
||
| def run_tests(self, generate_html: bool = True, generate_xml: bool = True) -> bool: | ||
| """Run pytest with coverage. | ||
|
|
||
| Args: | ||
| generate_html: Whether to generate HTML report | ||
| generate_xml: Whether to generate XML report | ||
|
|
||
| Returns: | ||
| True if tests passed, False otherwise | ||
| """ | ||
| cmd = [ | ||
| "poetry", "run", "pytest", | ||
| f"--cov={self.module_name}", | ||
| "--cov-report=term", | ||
| ] | ||
|
|
||
| if generate_html: | ||
| cmd.append("--cov-report=html") | ||
| if generate_xml: | ||
| cmd.append("--cov-report=xml") | ||
|
|
||
| result = subprocess.run(cmd, capture_output=False) | ||
| return result.returncode == 0 | ||
|
|
||
| def get_coverage_report(self) -> str: | ||
| """Get the coverage report output.""" | ||
| result = subprocess.run( | ||
| ["poetry", "run", "coverage", "report"], | ||
| capture_output=True, | ||
| text=True, | ||
| ) | ||
| return result.stdout | ||
|
|
||
| def get_coverage_report_with_missing(self) -> str: | ||
| """Get the coverage report with missing lines.""" | ||
| result = subprocess.run( | ||
| ["poetry", "run", "coverage", "report", "--show-missing"], | ||
| capture_output=True, | ||
| text=True, | ||
| ) | ||
| return result.stdout | ||
|
|
||
| def parse_coverage(self, report: str) -> CoverageResult: | ||
| """Parse coverage report and extract metrics. | ||
|
|
||
| Args: | ||
| report: The coverage report output | ||
|
|
||
| Returns: | ||
| CoverageResult with parsed metrics | ||
| """ | ||
| total_coverage = 0.0 | ||
| by_module: dict[str, float] = {} | ||
| low_coverage_files: list[tuple[str, float]] = [] | ||
| no_coverage_files: list[str] = [] | ||
|
|
||
| lines = report.strip().split("\n") | ||
|
|
||
| for line in lines: | ||
| # Parse TOTAL line | ||
| if line.startswith("TOTAL"): | ||
| match = re.search(r"(\d+(?:\.\d+)?)%", line) | ||
| if match: | ||
| total_coverage = float(match.group(1)) | ||
| continue | ||
|
|
||
| # Parse module lines | ||
| if self.module_name in line: | ||
| parts = line.split() | ||
| if len(parts) >= 4: | ||
| file_path = parts[0] | ||
| # Extract coverage percentage | ||
| match = re.search(r"(\d+(?:\.\d+)?)%", line) | ||
| if match: | ||
| coverage = float(match.group(1)) | ||
| by_module[file_path] = coverage | ||
|
|
||
| if coverage == 0: | ||
| no_coverage_files.append(file_path) | ||
| elif coverage < self.low_coverage_threshold: | ||
| low_coverage_files.append((file_path, coverage)) | ||
|
|
||
| return CoverageResult( | ||
| total_coverage=total_coverage, | ||
| by_module=by_module, | ||
| low_coverage_files=sorted(low_coverage_files, key=lambda x: x[1]), | ||
| no_coverage_files=no_coverage_files, | ||
| success=total_coverage >= self.threshold, | ||
| ) | ||
|
|
||
| def print_header(self, title: str) -> None: | ||
| """Print a formatted header.""" | ||
| print("=" * 42) | ||
| print(f"CodeFlow Engine - {title}") | ||
| print("=" * 42) | ||
| print() | ||
|
|
||
| def print_section(self, title: str) -> None: | ||
| """Print a formatted section header.""" | ||
| print(title) | ||
| print("-" * 40) | ||
|
|
||
| def check_coverage(self) -> int: | ||
| """Check if coverage meets the threshold. | ||
|
|
||
| Returns: | ||
| Exit code (0 for success, 1 for failure) | ||
| """ | ||
| self.print_header("Coverage Check") | ||
|
|
||
| # Check if coverage file exists | ||
| if not Path("coverage.xml").exists(): | ||
| print("Coverage file not found. Running tests with coverage...") | ||
| if not self.run_tests(): | ||
| print("Tests failed!") | ||
| return 1 | ||
| print() | ||
|
|
||
| report = self.get_coverage_report() | ||
| result = self.parse_coverage(report) | ||
|
|
||
| print(f"Current coverage: {result.total_coverage}%") | ||
| print(f"Target coverage: {self.threshold}%") | ||
| print() | ||
|
|
||
| if result.success: | ||
| print(f"Coverage {result.total_coverage}% meets threshold of {self.threshold}%") | ||
| return 0 | ||
| else: | ||
| print(f"Coverage {result.total_coverage}% is below threshold of {self.threshold}%") | ||
| print() | ||
| self.print_section("Coverage by module:") | ||
| report_with_missing = self.get_coverage_report_with_missing() | ||
| for line in report_with_missing.split("\n"): | ||
| if self.module_name in line: | ||
| print(line) | ||
| return 1 | ||
|
|
||
| def measure_coverage(self) -> int: | ||
| """Measure coverage and generate detailed report. | ||
|
|
||
| Returns: | ||
| Exit code (always 0) | ||
| """ | ||
| self.print_header("Coverage Measurement") | ||
|
|
||
| print("Running tests with coverage...") | ||
| tests_passed = self.run_tests(generate_html=True, generate_xml=True) | ||
| if not tests_passed: | ||
| print("Warning: Some tests failed. Coverage report may be incomplete.") | ||
| print() | ||
|
|
||
| self.print_header("Coverage Summary") | ||
|
|
||
| report = self.get_coverage_report() | ||
| result = self.parse_coverage(report) | ||
|
|
||
| print(f"Overall Coverage: {result.total_coverage}%") | ||
| print() | ||
|
|
||
| self.print_section("Coverage by Module (sorted by coverage %):") | ||
| report_with_missing = self.get_coverage_report_with_missing() | ||
| module_lines = [ | ||
| line for line in report_with_missing.split("\n") | ||
| if self.module_name in line | ||
| ] | ||
| for line in sorted(module_lines, key=lambda x: self._extract_coverage(x)): | ||
| print(line) | ||
| print() | ||
|
|
||
| self.print_section(f"Files with Coverage < {self.low_coverage_threshold}%:") | ||
| if result.low_coverage_files: | ||
| for file_path, coverage in result.low_coverage_files: | ||
| print(f" {file_path}: {coverage}%") | ||
| else: | ||
| print("None") | ||
| print() | ||
|
|
||
| self.print_section("Files with No Coverage:") | ||
| if result.no_coverage_files: | ||
| for file_path in result.no_coverage_files: | ||
| print(f" {file_path}") | ||
| else: | ||
| print("None") | ||
| print() | ||
|
|
||
| self.print_header("Detailed Report") | ||
| print("HTML report generated: htmlcov/index.html") | ||
| print("XML report generated: coverage.xml") | ||
| print() | ||
| print("Open HTML report:") | ||
| print(" - macOS/Linux: open htmlcov/index.html") | ||
| print(" - Windows: start htmlcov/index.html") | ||
| print() | ||
|
|
||
| return 0 | ||
|
|
||
| def _extract_coverage(self, line: str) -> float: | ||
| """Extract coverage percentage from a report line.""" | ||
| match = re.search(r"(\d+(?:\.\d+)?)%", line) | ||
| return float(match.group(1)) if match else 0.0 | ||
|
|
||
|
|
||
| def main() -> int: | ||
| """Main entry point for the coverage CLI.""" | ||
| parser = argparse.ArgumentParser( | ||
| description="Unified coverage tool for test quality gates" | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| "command", | ||
| choices=["check", "measure"], | ||
| help="Command to run: 'check' validates threshold, 'measure' generates full report", | ||
| ) | ||
| parser.add_argument( | ||
| "--threshold", | ||
| type=float, | ||
| default=70.0, | ||
| help="Coverage threshold percentage (default: 70)", | ||
| ) | ||
| parser.add_argument( | ||
| "--module", | ||
| type=str, | ||
| default="codeflow_engine", | ||
| help="Module name to measure coverage for (default: codeflow_engine)", | ||
| ) | ||
| parser.add_argument( | ||
| "--low-threshold", | ||
| type=float, | ||
| default=50.0, | ||
| help="Threshold for flagging low coverage files (default: 50)", | ||
| ) | ||
|
|
||
| args = parser.parse_args() | ||
|
|
||
| runner = CoverageRunner( | ||
| module_name=args.module, | ||
| threshold=args.threshold, | ||
| low_coverage_threshold=args.low_threshold, | ||
| ) | ||
|
|
||
| if args.command == "check": | ||
| return runner.check_coverage() | ||
| elif args.command == "measure": | ||
| return runner.measure_coverage() | ||
|
|
||
| return 1 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.