From 90b26c7f3ce255848f815a8be568c8259d230f56 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 23 Feb 2026 13:50:41 +1100 Subject: [PATCH 1/3] fix: code review improvements - docs, scripts, and tooling - Rewrite corrupted docs/testing.md (1275 -> 222 lines, deduplicated) - Rewrite corrupted tests/README.md (357 -> 97 lines, references only existing files) - Update CONTRIBUTING.md from Python template to shell-project instructions - Remove no-op shift statements in generate-report.sh arg parsing - Fix misleading GITHUB_OUTPUT error message in CLI mode - Fix docs/hyperlinks.md stating '> 1' when behavior is '> 0' - Add .tmp/ file staging guidelines to copilot-instructions.md - Add .tmp/ to .gitignore - Update CHANGELOG.md with all fixes --- .github/copilot-instructions.md | 29 +- .gitignore | 3 + CHANGELOG.md | 12 + CONTRIBUTING.md | 55 +- docs/hyperlinks.md | 2 +- docs/testing.md | 1268 +++---------------------------- generate-report.sh | 10 +- tests/README.md | 339 +-------- 8 files changed, 228 insertions(+), 1490 deletions(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index fe8fc06..dfcd487 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -86,13 +86,34 @@ root/ - Be specific and descriptive - Reference issues when applicable +## File Editing Guidelines + +### ⚠️ Critical: Use `.tmp/` for File Staging + +**Problem:** CLI tools like `gh` are more reliable when given a file as input rather than piped/escaped text. Shell escaping for multi-line content (release notes, issue bodies, PR descriptions) is fragile and error-prone. + +**Solution:** Always use the `create_file` tool to write content to `.tmp/`, then pass the file to CLI tools: + +``` +1. create_file -> .tmp/my-replacement.md (use create_file tool) +2. terminal -> cp .tmp/my-replacement.md docs/my-file.md +3. terminal -> rm .tmp/my-replacement.md (optional cleanup) +``` + +**Rules:** +- `.tmp/` is in `.gitignore` — never committed +- Never use heredoc (`<< 'EOF'`) in terminal for multi-line file content +- Never use `echo` with shell escaping for file creation +- Use `create_file` tool for new files, `replace_string_in_file` for edits +- For full file replacements: create in `.tmp/`, then `cp` over the target + ## GitHub CLI (`gh`) Usage -### ⚠️ Critical: Use Temporary Files for Output +### ⚠️ Critical: Use Temporary Files for Input and Output -**Problem:** `gh` commands display interactive/formatted text that can clutter terminal output +**Problem:** `gh` commands are more reliable when given files as input (e.g., `--notes-file`, `--body-file`) rather than inline text with shell escaping. Output can also clutter the terminal with interactive formatting. -**Solution:** Always redirect `gh` output to temporary files: +**Solution:** Always use `.tmp/` files for `gh` CLI input, and redirect output to temporary files: ```bash # ❌ BAD - Output may be cluttered with interactive display @@ -128,7 +149,7 @@ echo "Release notes here" > /tmp/notes.md gh release create v1.0.0 --notes-file /tmp/notes.md > /tmp/result.txt 2>&1 ``` -**Why:** Prevents interactive prompts, formatting codes, and progress indicators from cluttering output +**Why:** Prevents interactive prompts, formatting codes, and progress indicators from cluttering output. File-based input avoids shell escaping issues with multi-line content. ## Development Workflow diff --git a/.gitignore b/.gitignore index abca477..2b2b24a 100644 --- a/.gitignore +++ b/.gitignore @@ -15,3 +15,6 @@ weekly-report.md # Test outputs test-output/ + +# Working directory for file staging +.tmp/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 6d26650..ab32f51 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed +- **Corrupted documentation**: Rewrote `docs/testing.md` (was 1275 lines of duplicated/interleaved content, now clean 222 lines) +- **Corrupted tests README**: Rewrote `tests/README.md` (was 357 lines of duplicated content, now clean 97 lines referencing only existing test files) +- **CLI error message**: Changed misleading `ERROR: GITHUB_OUTPUT environment variable not set!` to a debug message in CLI mode (GITHUB_OUTPUT is only relevant in GitHub Actions) +- **Hyperlinks documentation**: Fixed `docs/hyperlinks.md` overview that incorrectly said "metrics greater than 1" when the actual behavior is "metrics greater than 0" + +### Changed +- **CONTRIBUTING.md**: Updated from generic Python template (referenced `pip install`, `pytest`, PEP 8) to accurate shell-project instructions with correct tooling (`bash`, `curl`, `jq`) and test commands +- **Argument parsing**: Removed no-op `shift` statements inside `for arg in "$@"` loop in `generate-report.sh` (shift has no effect inside a for loop) +- **Copilot instructions**: Added `.tmp/` file staging guidelines to prevent heredoc/shell escaping issues +- **`.gitignore`**: Added `.tmp/` working directory for file staging + ## [2.2.0] - 2025-10-23 ### Added diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e9c1411..7150f36 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,4 @@ -# Contributing to QuantEcon GitHub Actions +# Contributing to QuantEcon Weekly Report Action Thank you for your interest in contributing! We welcome contributions from the community. @@ -16,34 +16,55 @@ Thank you for your interest in contributing! We welcome contributions from the c ```bash # Clone your fork -git clone https://github.com/YOUR-USERNAME/REPO-NAME.git -cd REPO-NAME +git clone https://github.com/YOUR-USERNAME/action-weekly-report.git +cd action-weekly-report -# Install development dependencies (if applicable) -pip install -r requirements-dev.txt +# Ensure dependencies are available +# Required: bash 4.0+, curl, jq, date +brew install jq # macOS +# sudo apt-get install jq # Linux -# Run tests -pytest tests/ +# Run basic validation tests (no token needed) +./tests/test-basic.sh + +# Run with real data (requires GitHub token) +export GITHUB_TOKEN=ghp_xxxxx +./generate-report.sh --org=QuantEcon +cat report.md ``` ## Code Style -- Follow PEP 8 for Python code -- Use meaningful variable and function names -- Add docstrings for functions and classes -- Keep functions small and focused +### Shell Scripts +- Use `set -e` for error handling +- Add comments for complex logic +- Use descriptive variable names +- Add function documentation +- Include debug output for troubleshooting + +### Documentation +- Use clear headings (H1, H2, H3) +- Include code examples +- Use proper markdown formatting +- Keep tables concise and readable + +### Commit Messages +- Use conventional commits: `feat:`, `fix:`, `docs:`, `test:`, `chore:` +- Be specific and descriptive +- Reference issues when applicable ## Testing -- Write tests for new functionality -- Ensure existing tests pass -- Test with different Python versions if applicable -- Include integration tests for the GitHub Action +- Run `./tests/test-basic.sh` for syntax and structure validation +- Run `./test-hyperlinks.sh` to test the hyperlink feature +- Run `./tests/test-external-repos.sh` to test external repo format validation +- For live testing: `./generate-report.sh --token=ghp_xxx --org=QuantEcon` +- See [docs/testing.md](docs/testing.md) for the full testing guide ## Pull Request Process 1. Update documentation if needed -2. Add entries to CHANGELOG.md +2. Add entries to CHANGELOG.md under `[Unreleased]` 3. Ensure CI passes 4. Request review from maintainers @@ -54,7 +75,7 @@ When reporting issues, please include: - Clear description of the problem - Steps to reproduce - Expected vs actual behavior -- Environment details (OS, Python version, etc.) +- Environment details (OS, shell version, etc.) - Relevant logs or error messages ## Questions? diff --git a/docs/hyperlinks.md b/docs/hyperlinks.md index 2711473..cc85ce2 100644 --- a/docs/hyperlinks.md +++ b/docs/hyperlinks.md @@ -2,7 +2,7 @@ ## Overview -The weekly report now includes **clickable hyperlinks** for metrics greater than 1. When viewing the report in Markdown, clicking on a number will take you directly to the relevant GitHub page showing those specific items. +The weekly report now includes **clickable hyperlinks** for metrics greater than 0. When viewing the report in Markdown, clicking on a number will take you directly to the relevant GitHub page showing those specific items. ## Feature Behavior diff --git a/docs/testing.md b/docs/testing.md index 89b81a2..85ccb99 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -1,1274 +1,222 @@ -# Testing Guide - Activity Report Action# Testing Guide# Testing Guide# Testing Guide - Weekly Report Action - - +# Testing Guide ## Overview +This guide explains how to test the activity report action, both locally during development and in CI/CD. The action supports both GitHub Actions mode and standalone CLI mode. - -This guide explains how to test the activity report action, both locally during development and in production. The action supports both GitHub Actions mode and standalone CLI mode.## Quick Start - - - ---- - - - -## Quick Start### Basic Validation## Quick Start## Overview - - +## Quick Start ### 1. Basic Validation (No Token Required) - - -```bash```bash - +```bash # Run basic syntax and structure tests - -./tests/test-basic.sh# Run basic tests (no token required) - +./tests/test-basic.sh ``` -./tests/test-basic.sh### Basic ValidationThis guide explains how to test the weekly report action improvements, both locally during development and in production. - **What it checks:** - -- ✅ Script syntax is valid``` - -- ✅ Required functions exist - -- ✅ File structure is correct - -- ✅ No obvious errors - -### Generate Real Report +- Script syntax is valid +- Required functions exist +- File structure is correct +- No obvious errors **Run time:** ~5 seconds -```bash## Test Files Summary - ### 2. Show Help (No Token Required) ```bash - -```bash - -# Display all available options# Show help and options# Run basic tests (no token required) - ./generate-report.sh --help +``` -```./generate-report.sh --help - - - -**Shows:**./tests/test-basic.sh| Test | Token Required | Purpose | Run Time | - -- All CLI options - -- Usage examples# Generate report for last 7 days - -- Default values - -- Token requirementsexport GITHUB_TOKEN=ghp_xxxxx```|------|---------------|---------|----------| - - - -### 3. Generate Real Report (Token Required)./generate-report.sh - - +**Shows:** All CLI options, usage examples, default values, and token requirements. -```bash| `test-improvements.sh` | ❌ No | Validates code structure, syntax, and patterns | ~5 sec | +### 3. Generate Real Report (Token Required) +```bash # Using environment variable - -export GITHUB_TOKEN="ghp_xxxxx"# Or use command-line argument - +export GITHUB_TOKEN="ghp_xxxxx" ./generate-report.sh -./generate-report.sh --token=ghp_xxxxx### Generate Real Report| `test-show-report.sh` | ❌ No | Shows report format and example output | ~1 sec | - -# Using command-line argument - +# Or using command-line argument ./generate-report.sh --token=ghp_xxxxx - - -# View the generated report# View the report| `test-basic.sh` | ❌ No | Basic functionality validation | ~5 sec | - +# View the generated report cat report.md +``` -```cat report.md - - - -**What it does:**``````bash| `test-report-preview.sh` | ✅ Yes | Generates real report from live API | ~30-60 sec | - +**What it does:** - Validates token - - Fetches organization repos - - Collects metrics for active repos - -- Generates markdown report## What the Token Does# Show help and options - +- Generates markdown report - Saves to `report.md` (default) - - ---- - -**The GitHub token is ONLY used for:**./generate-report.sh --help## Recommended Testing Flow - -## What the Token Does - -- ✅ Reading repository data via GitHub API - -### Token Usage (Read-Only) - -- ✅ Fetching issues, PRs, and commits (read-only) +## Token Usage (Read-Only) **The GitHub token is ONLY used for:** - -- ✅ Reading repository data via GitHub API- ✅ Accessing organization information - -- ✅ Fetching issues, PRs, and commits (read-only) - -- ✅ Accessing organization information# Generate report for last 7 days### During Development (No Token Needed) - -- ✅ Checking rate limits - -**The token does NOT:** +- Reading repository data via GitHub API +- Fetching issues, PRs, and commits (read-only) +- Accessing organization information +- Checking rate limits **The token does NOT:** +- Create GitHub issues +- Post or publish anything +- Modify any repositories +- Write anything to GitHub -- ❌ Create GitHub issues- ❌ Create GitHub issuesexport GITHUB_TOKEN=ghp_xxxxx - -- ❌ Post or publish anything - -- ❌ Modify any repositories- ❌ Post or publish anything - -- ❌ Write anything to GitHub - -- ❌ Modify any repositories./generate-report.sh```bash - -### Required Scopes - -- ❌ Write anything to GitHub - -Your token needs: - -- `repo` - Access to repository data# 1. Validate your changes +**Required Scopes:** +- `repo` — Access to repository data +- `read:org` — Access to organization info -- `read:org` - Access to organization info - -**In CLI mode:** Token reads data → Script generates markdown file → Done - -### Token Security - -**In Action mode:** Token reads data → Script generates markdown file → Your workflow (optionally) posts it# Or use command-line argument./tests/test-improvements.sh - -- Token is validated on startup - -- Invalid tokens fail immediately - -- Clear error messages with actionable guidance - -- Rate limit checked and displayed## Testing Scenarios./generate-report.sh --token=ghp_xxxxx - - - ---- - - - -## Testing Scenarios### 1. Test Default Behavior# 2. See what the report looks like +**In CLI mode:** Token reads data -> Script generates markdown file -> Done. +**In Action mode:** Token reads data -> Script generates markdown file -> Your workflow (optionally) posts it. +## Testing Scenarios ### Test 1: Default Behavior - - -```bash```bash# View the report./tests/test-show-report.sh - -# Generate report for last 7 days - -./generate-report.sh --token=ghp_xxxxx./generate-report.sh --token=ghp_xxxxx - +```bash +./generate-report.sh --token=ghp_xxxxx ``` -```cat report.md``` - -**Expected output:** - -- Report for last 7 days - -- Saved to `report.md` - -- Includes all active repositories**Expected output:**``` - -- Shows token validation success - -- Displays repository processing progress- Report for last 7 days - - - -**Validation:**- Saved to `report.md`This gives you: +**Expected:** Report for last 7 days, saved to `report.md`, includes all active repositories. +**Validation:** ```bash - -# Check the report was created- Includes all active repositories - ls -lh report.md - -## Testing Scenarios- ✅ Syntax validation - -# View content - -cat report.md### 2. Test Custom Date Range - - - -# Check for expected sections- ✅ Structure verification - +cat report.md grep "Activity Report" report.md - -grep "Total Repositories" report.md```bash - +grep "Total Repositories" report.md ``` -# Specific period when action-translation-sync had 57 commits### 1. Test Default Behavior- ✅ Example output format - ### Test 2: Custom Date Range -./generate-report.sh --token=ghp_xxxxx --start=2025-10-16 --end=2025-10-18 - ```bash - -# Specific period when action-translation-sync had 57 commits```- ✅ Quick feedback loop - ./generate-report.sh --token=ghp_xxxxx --start=2025-10-16 --end=2025-10-18 - ``` +**Expected:** Report for Oct 16-18 only. Should include repos with activity in that window. +### Test 3: Custom Output File -**Expected output:****Expected output:**```bash - -- Report for Oct 16-18, 2025 only - -- Should include action-translation-sync with ~57 commits- Report for specified dates only - -- Validates activity detection works correctly - -- Should include action-translation-sync with ~57 commits./generate-report.sh --token=ghp_xxxxx### Before Committing (Optional, Requires Token) - -**Validation:** - -```bash- Validates activity detection works correctly - -# Check for action-translation-sync - -grep "action-translation-sync" report.md``` - - - -# Verify commit count### 3. Test Custom Output File - -grep -A 1 "action-translation-sync" report.md | grep "57" - -``````bash - - - -### Test 3: Custom Output File```bash - - - -```bash./generate-report.sh --token=ghp_xxxxx --output=monthly-report.md**Expected output:**# Test with real data - -# Save to custom filename - -./generate-report.sh --token=ghp_xxxxx --output=monthly-report.mdcat monthly-report.md - - - -# Verify```- Report for last 7 daysexport GITHUB_TOKEN="your_github_token" - +```bash +./generate-report.sh --token=ghp_xxxxx --output=monthly-report.md cat monthly-report.md - ``` +**Expected:** Report saved to `monthly-report.md`. - -**Expected output:****Expected output:**- Saved to `report.md`./tests/test-report-preview.sh - -- Report saved to `monthly-report.md` - -- Standard report format- Report saved to `monthly-report.md` - -- All metrics included - -- Default filename not created- Includes all active repositories``` - -### Test 4: Custom Organization - - +### Test 4: Different Organization ```bash - -# Generate report for different org### 4. Test Different Organization - -./generate-report.sh --token=ghp_xxxxx --org=YourOrgName - +./generate-report.sh --token=ghp_xxxxx --org=YourOrg ``` - - -**Expected output:**```bash### 2. Test Custom Date RangeThis gives you: - -- Report title shows your organization name - -- Repositories from your organization./generate-report.sh --token=ghp_xxxxx --org=YourOrg - -- Activity metrics for your repos - -```- ✅ Real repository data +**Expected:** Report title shows your organization name; repositories from your org. ### Test 5: Exclude Repositories - - ```bash +./generate-report.sh --token=ghp_xxxxx --exclude="lecture-.*\.notebooks" +``` -# Exclude specific repos**Expected output:**```bash- ✅ Actual API integration - -./generate-report.sh --token=ghp_xxxxx --exclude=archived-repo,test-repo - -```- Report for specified organization - - - -**Expected output:**- Different repository list# Specific period when action-translation-sync had 57 commits- ✅ Live activity tracking - -- Report excludes specified repos - -- Other active repos included normally - -- Excluded repos not in debug output +**Expected:** Matching repositories excluded from report. Debug output shows which repos were excluded and which patterns matched. -### 5. Test with Exclusions./generate-report.sh --token=ghp_xxxxx --start=2025-10-16 --end=2025-10-18- ✅ Production-like results +### Test 6: External Repositories -### Test 6: API Rate Limiting +```bash +./generate-report.sh --token=ghp_xxxxx --track-external-repos=executablebooks/sphinx-proof,executablebooks/sphinx-exercise +``` +**Expected:** Separate "External Repositories" section in report with independent totals. +### Test 7: API Rate Limiting ```bash - -# Add delay between API calls```bash``` - ./generate-report.sh --token=ghp_xxxxx --delay=1 +``` -```./generate-report.sh --token=ghp_xxxxx --exclude=repo1,repo2 - - - -**Expected output:**```## Detailed Test Descriptions - -- Slower execution (1 second between repos) - -- All metrics collected correctly - -- Useful for avoiding rate limits - -**Expected output:****Expected output:** - -### Test 7: Quiet Day (No Activity) +**Expected:** Slower execution (1 second between repos), all metrics collected correctly. -- Specified repositories excluded from report +### Test 8: Quiet Day (No Activity) ```bash - -# Test a Sunday or quiet period- Reduced repository count- Report for specified dates only### 1. test-improvements.sh - Code Validation - ./generate-report.sh --token=ghp_xxxxx --start=2025-10-05 --end=2025-10-05 - ``` +**Expected:** Report shows 0 repositories with activity. No wasteful API calls. +## Testing in GitHub Actions -**Expected output:**## What Gets Tested- Should include action-translation-sync with ~57 commits +The CI workflow (`.github/workflows/ci.yml`) runs three test jobs automatically: -- Report shows 0 repositories with activity +1. **test-script** — Syntax validation + `test-basic.sh` +2. **test-action** — Live report generation using the composite action +3. **test-dry-run** — Graceful failure with invalid token -- No wasteful API calls (no fallback to all repos) +## Validation Checklist -- Clean empty report or clear message - -### test-basic.sh- Validates activity detection works correctly**Purpose:** Validates the improvements made to fix repository discovery +When testing, verify: -### Test 8: macOS Compatibility +- [ ] Report includes repositories with various activity types (commits, PRs, issues) +- [ ] All metric columns present: Current Issues, Opened Issues, Closed Issues, Opened PRs, Merged PRs, Commits +- [ ] Totals row at bottom of table +- [ ] Summary statistics in Details section +- [ ] Clickable hyperlinks for non-zero metrics +- [ ] Zero-activity repos excluded from report +- [ ] CLI options work: `--help`, `--token`, `--start`, `--end`, `--output`, `--org`, `--exclude` +## Troubleshooting +### "GITHUB_TOKEN is required" ```bash - -# Run on macOS (uses BSD date and head)Validates: - +export GITHUB_TOKEN=ghp_xxxxx +# or ./generate-report.sh --token=ghp_xxxxx +``` -```- ✅ Script structure and syntax - - - -**Expected output:**- ✅ Error handling### 3. Test Custom Output File**What it checks:** - -- Works identically to Linux - -- Date parsing successful- ✅ Environment variable processing - -- Text processing correct - -- Report format identical- ✅ Basic functionality- ✅ Required dependencies (curl, jq, date) - - - ---- - - - -## Testing in GitHub Actions**Run time:** ~5 seconds ```bash- ✅ Bash script syntax - - - -### Basic Workflow Test**Token required:** No - - - -```yaml./generate-report.sh --token=ghp_xxxxx --output=monthly-report.md- ✅ Required functions and variables - -name: Test Weekly Report - -on:### Direct Script Execution - - workflow_dispatch: - -cat monthly-report.md- ✅ New columns (Opened PRs, Commits) - -jobs: - - test:Tests: - - runs-on: ubuntu-latest - - steps:- ✅ Command-line argument parsing```- ✅ Correct API endpoints - - - uses: actions/checkout@v3 - - - ✅ Date range handling - - - name: Generate activity report - - uses: QuantEcon/action-weekly-report@v2- ✅ API integration- ✅ Old search API removed - - with: - - github-token: ${{ secrets.GITHUB_TOKEN }}- ✅ Report generation - - - - - name: Display report- ✅ Real repository data**Expected output:**- ✅ Date calculations - - run: cat report.md +### "command not found" +```bash +chmod +x generate-report.sh +chmod +x tests/*.sh ``` +### "command not found: jq" +```bash +# macOS +brew install jq -### Custom Date Range Test**Run time:** ~30-60 seconds - Report saved to `monthly-report.md` - +# Linux +sudo apt-get install jq +``` +### Date format errors on macOS -```yaml**Token required:** Yes (read-only access) +The script handles both GNU date (Linux) and BSD date (macOS). Use ISO format: `YYYY-MM-DD`. -- name: Generate monthly report +### Empty or incomplete report - uses: QuantEcon/action-weekly-report@v2- Default filename not created**Run:** +Check: +1. Token has correct permissions (`repo`, `read:org`) +2. Organization name is correct +3. Date range has activity +4. No rate limiting (add `--delay=1`) - with: +### Rate limit exceeded - github-token: ${{ secrets.GITHUB_TOKEN }}## Validation Checklist - - start-date: '2025-10-01' - - end-date: '2025-10-31'```bash - -``` - -When testing, verify: - -### Multiple Organizations Test - -### 4. Test Different Organization./tests/test-improvements.sh - -```yaml - -- name: Generate report for Org 1- [ ] Report includes repositories with various activity types: - - uses: QuantEcon/action-weekly-report@v2 - - with: - Repositories with only commits``` - - github-token: ${{ secrets.GITHUB_TOKEN }} - - organization: 'OrgName1' - Repositories with only PRs - - - -- name: Generate report for Org 2 - Repositories with only issues```bash - - uses: QuantEcon/action-weekly-report@v2 - - with: - Repositories with mixed activity - - github-token: ${{ secrets.GITHUB_TOKEN }} - - organization: 'OrgName2'./generate-report.sh --token=ghp_xxxxx --org=YourOrg**Expected output:** - -``` - -- [ ] New metrics are present: - ---- - - - [ ] "Opened PRs" column`````` - -## Validation Testing - - - [ ] "Commits" column - -### Validate Against GitHub UI - - - [ ] "Current Issues" column========================================== - -Compare report metrics with GitHub's web interface: - - - -**For each repository in report:** - -- [ ] Previously missing repositories are included:**Expected output:**Weekly Report Action - Validation Tests - -1. **Current Issues** - - - Go to: `https://github.com/ORG/REPO/issues` - [ ] action-translation-sync (57 commits Oct 16-18) - - - Count open issues - - - Should match "Current Issues" column - [ ] Repositories with fork-based PRs- Report for specified organization========================================== - - - -2. **Opened Issues** - - - Filter: `is:issue created:2025-10-13..2025-10-20` - - - Count results- [ ] Report format is correct:- Different repository list - - - Should match "Opened Issues" column - - - [ ] Markdown table with proper headers - -3. **Closed Issues** - - - Filter: `is:issue closed:2025-10-13..2025-10-20` - [ ] Totals row at bottomTest 1: Checking required dependencies... - - - Count results - - - Should match "Closed Issues" column - [ ] Summary statistics - - - -4. **Opened PRs**### 5. Test with Exclusions ✓ curl is installed - - - Filter: `is:pr created:2025-10-13..2025-10-20` - - - Count results- [ ] CLI options work: - - - Should match "Opened PRs" column - - - [ ] `--help` shows usage ✓ jq is installed - -5. **Merged PRs** - - - Filter: `is:pr merged:2025-10-13..2025-10-20` - [ ] `--token` accepts token - - - Count results - - - Should match "Merged PRs" column - [ ] `--start` and `--end` set date range```bash ✓ date is installed - - - -6. **Commits** - [ ] `--output` changes filename - - - Go to: `https://github.com/ORG/REPO/commits/main` - - - Filter by date range - [ ] `--org` changes organization./generate-report.sh --token=ghp_xxxxx --exclude=repo1,repo2✓ Test 1 PASSED: All dependencies are installed - - - Count commits - - - Should match "Commits" column - - - -### Expected Results## GitHub Actions Integration``` - - - -All metrics should match GitHub UI **exactly** (100% accuracy). - - - -See [validation.md](validation.md) for detailed examples with screenshots.The action works automatically in workflows:... - - - ---- - - - -## Troubleshooting```yaml**Expected output:** - - - -### Token Validation Fails- name: Generate weekly report - - - -**Error:** `❌ ERROR: GitHub token validation failed (HTTP 401)` uses: QuantEcon/action-weekly-report@v2- Specified repositories excluded from report========================================== - - - -**Solutions:** with: - -1. Check token is valid: https://github.com/settings/tokens - -2. Verify scopes include `repo` and `read:org` github-token: ${{ secrets.GITHUB_TOKEN }}- Reduced repository countTest Summary - -3. Check token hasn't expired - -4. Try regenerating token organization: 'QuantEcon' - - - -### No Repositories Found```========================================== - - - -**Error:** `No repositories found with activity in date range` - - - -**Solutions:****What happens:**## What Gets TestedTests Passed: 7 - -1. Check date range is correct - -2. Verify organization name is correct1. Script runs with `INPUT_*` environment variables - -3. Confirm there was activity in that period - -4. Try expanding date range2. Report saved to `report.md`Tests Failed: 0 - - - -### Rate Limit Exceeded3. Outputs set for downstream steps - - - -**Error:** `API rate limit exceeded`4. Your workflow can optionally post the report as an issue using a separate action### test-basic.sh - - - -**Solutions:** - -1. Wait for rate limit reset (shown in error) - -2. Use `--delay=1` to slow down requests**No changes required** to existing workflow files - the script is backward compatible.✓ All tests passed! - -3. Check you're using authenticated token - -4. Use token with higher rate limit - - - -### macOS Date Parsing Fails## TroubleshootingValidates:``` - - - -**Error:** `date: invalid date` - - - -**Solutions:**### "GITHUB_TOKEN is required"- ✅ Script structure and syntax - -1. Ensure date format is `YYYY-MM-DD` - -2. Check script has BSD date fallback - -3. Verify date is valid (not future date) - -4. Try updating macOS if very old```bash- ✅ Error handling### 2. test-show-report.sh - Report Format Preview - - - -### Report Missing Repositories# Option 1: Environment variable - - - -**Check:**export GITHUB_TOKEN=ghp_xxxxx- ✅ Environment variable processing - -1. Repository had activity in date range? - -2. Repository in specified organization?./generate-report.sh - -3. Repository not in exclude list? - -4. Check debug output for filtering info- ✅ Basic functionality**Purpose:** Shows what the generated report will look like - - - -**Validation:**# Option 2: Command-line argument - -```bash - -# Run with debug to see all repos checked./generate-report.sh --token=ghp_xxxxx - -./generate-report.sh --token=xxx 2>&1 | grep "Processing" - -`````` - - - ----**Run time:** ~5 seconds **What it does:** - - - -## Performance Testing### "command not found" - - - -### Measure API Calls**Token required:** No- Shows existing `report.md` if present - - - -```bash```bash - -# Count API calls made - -./generate-report.sh --token=xxx 2>&1 | grep -c "curl"chmod +x generate-report.sh- Displays example report format if not - -``` - -chmod +x tests/*.sh - -**Expected for QuantEcon (209 repos, 9 active):** - -- Token validation: 2 calls```### Direct Script Execution- Validates report structure - -- Repo fetch: 3 calls (pagination) - -- Per-repo metrics: 9 × 3 = 27 calls - -- **Total: ~32 calls** (well under 5,000/hour limit) - -### Date format errors on macOS- Checks for new features - -### Measure Execution Time - - - -```bash - -# Time the report generationThe script automatically handles both GNU date (Linux) and BSD date (macOS).Tests: - -time ./generate-report.sh --token=xxx - -``` - - - -**Expected times:**If you see errors, use ISO format: `YYYY-MM-DD`- ✅ Command-line argument parsing**Run:** - -- Small org (<50 repos): 5-10 seconds - -- Medium org (50-150 repos): 10-30 seconds - -- Large org (150-250 repos): 30-60 seconds - -### Empty or incomplete report- ✅ Date range handling```bash - -With `--delay=1`: Add 1 second per active repository - - - ---- - -Check:- ✅ API integration./tests/test-show-report.sh - -## Automated Testing - -1. Token has correct permissions (`repo`, `read:org`) - -### CI/CD Integration - -2. Organization name is correct- ✅ Report generation``` - -```yaml - -name: Test Action3. Date range has activity - -on: [push, pull_request] - -4. No rate limiting (add `--delay=1`)- ✅ Real repository data - -jobs: - - test: - - runs-on: ubuntu-latest - - steps:## Creating a GitHub Token**Expected output:** - - - uses: actions/checkout@v3 - - - - - name: Run basic tests - - run: ./tests/test-basic.sh**What the token is used for:****Run time:** ~30-60 seconds ``` - - - - - name: Test report generation- Reading repository data (issues, PRs, commits) via GitHub API - - run: | - - ./generate-report.sh --token=${{ secrets.GITHUB_TOKEN }}- Reading organization information**Token required:** Yes========================================== - - test -f report.md - - grep -q "Activity Report" report.md- **NOT** for creating issues or modifying anything - -``` - -Weekly Report - Output Preview - -### Pre-commit Testing - -**How to create:** - -```bash - -#!/bin/bash## Validation Checklist========================================== - -# .git/hooks/pre-commit - -1. Go to https://github.com/settings/tokens - -# Run basic tests before commit - -./tests/test-basic.sh || exit 12. Click "Generate new token" → "Generate new token (classic)" - - - -echo "✓ Tests passed"3. Select scopes: - -``` - - - ✅ `repo` - Read repository data (or just `public_repo` for public orgs only)When testing, verify:📋 Expected Report Format - ---- - - - ✅ `read:org` - Read organization data - -## Summary - -4. Generate and copy token========================================== - -### Testing Checklist - -5. Use in commands or set as environment variable - -**Before Committing:** - -- [ ] Run `./tests/test-basic.sh` (passes)- [ ] Report includes repositories with various activity types: - -- [ ] Check script syntax is valid - -- [ ] Verify functions exist**Security note:** The token is only used to read data. The script does not create issues, post content, or modify repositories. Issue creation happens separately in your workflow (optional). - - - -**Before Releasing:** - Repositories with only commits# QuantEcon Weekly Report - -- [ ] Generate real report with CLI - -- [ ] Verify all metrics accurate## See Also - -- [ ] Test on both macOS and Linux - -- [ ] Validate token handling - Repositories with only PRs - -- [ ] Check error messages are clear - -- [ ] Test with multiple date ranges- [../README.md](../README.md) - Main usage documentation - -- [ ] Verify pagination works (>100 repos) - -- [ ] Cross-validate against GitHub UI- [improvements.md](improvements.md) - Technical implementation details - Repositories with only issues**Report Period:** October 11, 2025 - October 18, 2025 - - - -**In Production:**- [validation.md](validation.md) - Real-world validation examples - -- [ ] Monitor rate limit usage - -- [ ] Verify reports include all repos- [../tests/README.md](../tests/README.md) - Test file details - Repositories with mixed activity - -- [ ] Check metrics match expectations - -- [ ] Validate date ranges correct - -- [ ] Ensure zero-activity repos filtered## Summary - - - -### Quick Commands- [ ] New metrics are present: - - - -```bash - [ ] "Opened PRs" column| Repository | Current Issues | Opened Issues | Closed Issues | Opened PRs | Merged PRs | Commits | - -# Full test cycle - -./tests/test-basic.sh && \ - [ ] "Commits" column|------------|----------------|---------------|---------------|------------|------------|---------| - - ./generate-report.sh --token=xxx && \ - - cat report.md - [ ] "Current Issues" column| action-translation-sync | 0 | 0 | 0 | 0 | 0 | 57 | - - - -# Validate specific date range... - -./generate-report.sh --token=xxx --start=2025-10-16 --end=2025-10-18 - -- [ ] Previously missing repositories are included:``` - -# Check for specific repo - -./generate-report.sh --token=xxx | grep "action-translation-sync" - [ ] action-translation-sync (57 commits Oct 16-18) - -``` - - - [ ] Repositories with fork-based PRs### 3. test-report-preview.sh - Live API Test - ---- - - - -For more information: - -- **Technical Details:** [improvements.md](improvements.md)- [ ] Report format is correct:**Purpose:** Generates actual report from live GitHub data - -- **Validation Examples:** [validation.md](validation.md) - -- **Release Notes:** [releases/v2.0.0.md](releases/v2.0.0.md) - [ ] Markdown table with proper headers - - - - [ ] Totals row at bottom**Requirements:** - - - [ ] Summary statistics- GitHub personal access token - -- Token needs read access to: - -- [ ] CLI options work: - Organization repositories - - - [ ] `--help` shows usage - Repository issues - - - [ ] `--token` accepts token - Repository pull requests - - - [ ] `--start` and `--end` set date range - - - [ ] `--output` changes filename**Setup:** - - - [ ] `--org` changes organization```bash - -# Create token at: https://github.com/settings/tokens - -## GitHub Actions Integration# Required scopes: repo (all), read:org - - - -The action works automatically in workflows:export GITHUB_TOKEN="ghp_xxxxxxxxxxxxxxxxxxxxx" - -``` - -```yaml - -- name: Generate weekly report**Run:** - - uses: QuantEcon/action-weekly-report@v2```bash - - with:./tests/test-report-preview.sh - - github-token: ${{ secrets.GITHUB_TOKEN }}``` - - organization: 'QuantEcon' - -```**What it does:** - -1. Validates GitHub token is set - -**What happens:**2. Calls real GitHub API endpoints - -1. Script runs with `INPUT_*` environment variables3. Fetches actual repository data - -2. Report saved to `report.md`4. Generates complete report - -3. Outputs set for downstream steps5. Displays report in terminal - -4. Compatible with existing workflows6. Shows statistics and validation results - - - -**No changes required** to existing workflow files - the script is backward compatible.**Expected output:** - -``` - -## Troubleshooting========================================== - -Weekly Report Preview Test - -### "GITHUB_TOKEN is required"========================================== - - - -```bash✓ GitHub token found - -# Option 1: Environment variable - -export GITHUB_TOKEN=ghp_xxxxx========================================== - -./generate-report.shConfiguration - -========================================== - -# Option 2: Command-line argumentOrganization: QuantEcon - -./generate-report.sh --token=ghp_xxxxxAPI Delay: 1 seconds - -```Date Range: Last 7 days - - - -### "command not found"========================================== - -Generating Report... - -```bash========================================== - -chmod +x generate-report.sh - -chmod +x tests/*.shFetching all repositories for QuantEcon... - -```Total repositories found: 45 - -Repositories with activity in the last week: 12 - -### Date format errors on macOSProcessing repository: action-translation-sync - -... - -The script automatically handles both GNU date (Linux) and BSD date (macOS). - -========================================== - -If you see errors, use ISO format: `YYYY-MM-DD`Report Generated! - -========================================== - -### Empty or incomplete report - -📊 WEEKLY REPORT PREVIEW: - -Check:========================================== - -1. Token has correct permissions (`repo`, `read:org`)[Full report content displayed here] - -2. Organization name is correct========================================== - -3. Date range has activity - -4. No rate limiting (add `--delay=1`)📈 Quick Stats: - - - Repositories with activity: 12 - -## Creating a GitHub Token - Report saved to: /path/to/report.md - - - -1. Go to https://github.com/settings/tokens🔍 Validation Checks: - -2. Click "Generate new token" → "Generate new token (classic)" ✅ action-translation-sync is included in report - -3. Select scopes: ✅ Commits column present - - - ✅ `repo` (Full control of private repositories) ✅ Opened PRs column present - - - ✅ `read:org` (Read organization data) - -4. Generate and copy token✅ Test completed successfully! - -5. Use in commands or set as environment variable``` - - - -## See Also### 4. test-basic.sh - Basic Functionality - - - -- [../README.md](../README.md) - Main usage documentation**Purpose:** Legacy test for basic script validation - -- [improvements.md](improvements.md) - Technical implementation details - -- [validation.md](validation.md) - Real-world validation examples**Run:** - -- [../tests/README.md](../tests/README.md) - Test file details```bash - -./tests/test-basic.sh -``` - -## Understanding Report Output - -### Report Structure - -The generated `report.md` contains: - -1. **Header** - Report period and title -2. **Summary Table** - Repository-by-repository breakdown -3. **Totals Row** - Aggregated statistics -4. **Details Section** - Numeric summaries -5. **Warnings** - API issues if any -6. **Footer** - Generation timestamp - -### Key Metrics - -| Metric | Description | Why It Matters | -|--------|-------------|----------------| -| Current Issues | Open issues right now | Shows workload | -| Opened Issues | New issues this week | Shows new problems | -| Closed Issues | Issues resolved | Shows progress | -| Opened PRs | PRs created | Shows development activity | -| Merged PRs | PRs merged | Shows completed work | -| Commits | Direct commits | Shows coding activity | - -### What To Look For - -✅ **Good signs:** -- `action-translation-sync` appears in table -- Commit counts look reasonable -- All active repos included -- No rate limit warnings - -⚠️ **Potential issues:** -- Missing repositories you know had activity -- Zero commits for repos with known activity -- Rate limit warnings -- API error messages - -## Troubleshooting - -### "command not found: jq" - -Install jq: -```bash -# macOS -brew install jq - -# Linux -sudo apt-get install jq -``` - -### "Rate limit exceeded" - -Add API delay: ```bash -export INPUT_API_DELAY=2 # 2 seconds between calls -./tests/test-report-preview.sh -``` - -### "No repositories found with recent activity" - -This might mean: -1. Actually no activity in last 7 days -2. Date calculation issue (check system date) -3. API filtering too aggressive - -### Token Permission Errors - -Ensure token has these scopes: -- `repo` (all sub-scopes) -- `read:org` - -Create new token at: https://github.com/settings/tokens - -## CI/CD Integration - -The action runs these tests automatically: -- ✅ Syntax validation (test-improvements.sh) -- ✅ Code structure checks -- ✅ No live API calls in CI (security) - -## Production Testing - -After deployment, verify by: - -1. **Check next weekly report issue** - - Look for `action-translation-sync` - - Verify commit counts - - Check new columns present - -2. **Compare with GitHub UI** - - Manual spot-checks of commit counts - - Verify issue/PR numbers match - - Confirm no active repos missing - -3. **Monitor for warnings** - - Rate limit messages - - API errors - - Incomplete data warnings - -## Quick Reference - -```bash -# Fast development cycle (no token) -./tests/test-improvements.sh && ./tests/test-show-report.sh - -# Full validation (requires token) -export GITHUB_TOKEN="your_token" -./tests/test-report-preview.sh - -# Clean up -rm -f report.md +# Add delay between API calls +./generate-report.sh --token=ghp_xxxxx --delay=1 ``` -## Next Steps +## See Also -1. Run validation: `./tests/test-improvements.sh` -2. Check format: `./tests/test-show-report.sh` -3. Optional live test: Set token and run `test-report-preview.sh` -4. Commit changes -5. Wait for next scheduled report -6. Verify results in production +- [../README.md](../README.md) — Main usage documentation +- [improvements.md](improvements.md) — Technical implementation details +- [validation.md](validation.md) — Real-world validation examples +- [hyperlinks.md](hyperlinks.md) — Clickable metrics feature +- [../tests/README.md](../tests/README.md) — Test file details diff --git a/generate-report.sh b/generate-report.sh index 7adbece..1dce81f 100755 --- a/generate-report.sh +++ b/generate-report.sh @@ -69,35 +69,27 @@ for arg in "$@"; do case $arg in --token=*) CLI_TOKEN="${arg#*=}" - shift ;; --org=*) CLI_ORG="${arg#*=}" - shift ;; --start=*) START_DATE="${arg#*=}" - shift ;; --end=*) END_DATE="${arg#*=}" - shift ;; --exclude=*) CLI_EXCLUDE="${arg#*=}" - shift ;; --track-external-repos=*) CLI_EXTERNAL_REPOS="${arg#*=}" - shift ;; --delay=*) CLI_DELAY="${arg#*=}" - shift ;; --output=*) CLI_OUTPUT="${arg#*=}" - shift ;; --help) show_usage @@ -819,7 +811,7 @@ if [ -n "$GITHUB_OUTPUT" ]; then echo "DEBUG: Outputs written to GITHUB_OUTPUT" echo "DEBUG: GITHUB_OUTPUT file size: $(wc -c < "$GITHUB_OUTPUT")" else - echo "ERROR: GITHUB_OUTPUT environment variable not set!" + echo "DEBUG: GITHUB_OUTPUT not set (expected in CLI mode, outputs are for GitHub Actions only)" fi echo "Report generated successfully!" diff --git a/tests/README.md b/tests/README.md index 2b195fe..a318171 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,314 +1,68 @@ -# Tests# Test Files for Weekly Report Action# Test Files for Weekly Report Action +# Tests +## Available Test Files +### test-basic.sh -## test-basic.sh - - - -Basic validation test that checks:This directory contains test scripts for the weekly-report action.This directory contains test scripts used to validate the weekly-report action functionality. - +Basic validation test that checks: - Script structure and syntax - - Error handling - - Environment variable processing - -- Mock API response handling## Available Tests## Test Files - - +- Mock API response handling **Run:** `./tests/test-basic.sh` +**Token required:** No +**Run time:** ~5 seconds +### test-external-repos.sh +Tests for the external repositories feature: +- Validates `org/repo` format checking +- Tests invalid format detection +- Tests mixed valid/invalid inputs +- Verifies empty external repos handling -## Development Testing### test-basic.sh### test-basic.sh - +**Run:** `./tests/test-external-repos.sh` +**Token required:** No (uses dummy token for format validation only) +### test-hyperlinks.sh (root) -For testing with real data, run the script directly from the project root:Basic functionality test that validates:- Shell script that tests the basic functionality of the weekly report generator +Tests for the clickable hyperlink feature: +- Validates `format_metric()` function behavior +- Verifies links are generated for counts > 0 +- Verifies plain text for counts = 0 +- Generates sample markdown table +**Run:** `./test-hyperlinks.sh` +**Token required:** No +## Development Testing -```bash- Script structure and error handling- Creates mock environment and API responses for testing +For testing with real data, run the script directly from the project root: +```bash # Show help and options +./generate-report.sh --help -./generate-report.sh --help- Environment variable processing- Validates script structure and error handling - - - -# Test with last 7 days (default)- Mock API response handling- Tests the generate-report.sh script without requiring real GitHub API access - +# Test with last 7 days (default) ./generate-report.sh --token=ghp_xxxxx -- Basic execution without real GitHub API access - # Test specific date range +./generate-report.sh --token=ghp_xxxxx --start=2025-10-16 --end=2025-10-18 -./generate-report.sh --token=ghp_xxxxx --start=2025-10-16 --end=2025-10-18### test-improvements.sh - - - -# View generated report**Run:** `./tests/test-basic.sh`- Validates all improvements made to fix repository discovery issues - +# View generated report cat report.md - -```- Checks for required functions, variables, and API endpoints - - - -See [../README.md](../README.md) for complete usage examples.### test-show-report.sh- Verifies new columns (Opened PRs, Commits) are present - - - -## CI IntegrationDemonstrates direct script execution:- Confirms old search API has been removed - - - -The GitHub Actions workflow automatically runs:- Shows usage examples if no token is available- Tests syntax and date calculations - -- `test-basic.sh` - Validates script structure and basic functionality - -- Shell script syntax checks- Generates real report if `GITHUB_TOKEN` is set- **Run this after making changes to ensure nothing broke** - -- Additional validation as needed - -- Displays the generated markdown output - -## Requirements - -- No API calls if token is not provided### test-show-report.sh - -- **Bash** 4.0+ - -- **jq** (for JSON parsing)- **Shows the generated report markdown output (NO TOKEN NEEDED)** - -- **curl** (for API calls) - -- **date** (for date calculations)**Run:** `./tests/test-show-report.sh`- Displays the report.md file if it exists - -- **GitHub Token** (for real API calls) - -- Shows expected report format if no report exists - -## Troubleshooting - -**With Token:** `GITHUB_TOKEN=ghp_xxx ./tests/test-show-report.sh`- Validates report structure and new columns - -### "command not found" - -```bash- **Use this to see what will be posted to GitHub Issues** - -chmod +x tests/*.sh - -chmod +x generate-report.sh## Running the Script Directly- **Recommended for quick checks without API access** - ``` +See [../README.md](../README.md) for complete CLI usage examples. +## CI Integration -### "GITHUB_TOKEN is required" - -```bashThe `generate-report.sh` script can now be run directly from the command line:### test-report-preview.sh - -export GITHUB_TOKEN=ghp_xxxxx - -# or- **Generates a real report from live GitHub API (REQUIRES TOKEN)** - -./generate-report.sh --token=ghp_xxxxx - -```### Basic Usage- Requires: `GITHUB_TOKEN` environment variable - - - -### Date format errors- Shows actual repository activity data from last 7 days - -Use ISO format: `YYYY-MM-DD` - -```bash```bash- Calls real GitHub API to fetch current data - -./generate-report.sh --start=2025-10-16 --end=2025-10-18 - -```# Set environment variable- Useful for testing with actual organization data - - - -## See Alsoexport GITHUB_TOKEN=ghp_xxxxx- **Use this when you need to test with real API data** - - - -- [../README.md](../README.md) - Main documentation with usage examples./generate-report.sh - -- [../docs/testing.md](../docs/testing.md) - Comprehensive testing guide - -- [../docs/improvements.md](../docs/improvements.md) - Technical details## Usage in CI - - -# Or use command line argument - -./generate-report.sh --token=ghp_xxxxxThese files are automatically used by the GitHub Actions CI workflow to test: - -```- Shell script syntax validation - -- Basic script execution and error handling - -### Custom Date Ranges- Environment variable processing - -- Mock API response handling - -```bash - -# Specific date range## Running Tests Locally - -./generate-report.sh --token=ghp_xxxxx --start=2025-10-01 --end=2025-10-07 - -### Quick Tests (No GitHub Token Required) - -# Start date only (will use 7 days from start) - -./generate-report.sh --token=ghp_xxxxx --start=2025-10-01```bash - -```# 1. Quick validation test - -chmod +x tests/test-improvements.sh - -### Different Organization./tests/test-improvements.sh - - - -```bash# 2. RECOMMENDED: Show report format and output - -./generate-report.sh --token=ghp_xxxxx --org=YourOrgchmod +x tests/test-show-report.sh - -```./tests/test-show-report.sh - - - -### Help# 3. Basic functionality test - -chmod +x tests/test-basic.sh - -```bash./tests/test-basic.sh - -./generate-report.sh --help - -```# 4. Test script syntax - -bash -n generate-report.sh - -## Development Workflow``` - - - -### Quick Development Test### Full Test with Real Data (Requires GitHub Token) - - - -```bash```bash - -# 1. Basic syntax check# Generate and preview actual report from live API - -./tests/test-basic.shexport GITHUB_TOKEN="your_github_token_here" - -chmod +x tests/test-report-preview.sh - -# 2. See usage examples./tests/test-report-preview.sh - -./tests/test-show-report.sh``` - - - -# 3. Generate real report (requires token)### Quick Development Workflow - -export GITHUB_TOKEN=ghp_xxxxx - -./generate-report.sh --start=2025-10-16 --end=2025-10-18When developing and testing changes: - -``` - -```bash - -### Testing with Real Data# 1. Make your changes to generate-report.sh - - - -```bash# 2. Validate syntax and structure (no token needed) - -# Set your token./tests/test-improvements.sh - -export GITHUB_TOKEN=ghp_xxxxx - -# 3. See what the report will look like (no token needed) - -# Test last 7 days (default)./tests/test-show-report.sh - -./generate-report.sh --org=QuantEcon - -# 4. Optional: Test with real API data (requires token) - -# Test specific period (e.g., when action-translation-sync had 57 commits)export GITHUB_TOKEN="your_token" - -./generate-report.sh --org=QuantEcon --start=2025-10-16 --end=2025-10-18./tests/test-report-preview.sh - - - -# View the generated report# 5. Review the generated report - -cat report.mdcat report.md - -`````` - - - -## What Gets Tested## Test Requirements - - - -### test-basic.shThe tests require: - -- ✅ Script has proper shebang and error handling- bash shell - -- ✅ Required variables are defined- Standard Unix utilities (date, etc.) - -- ✅ Functions work correctly- GitHub CLI is mocked in the test environment - -- ✅ Error messages are clear - -## Real Testing - -### test-show-report.sh - -- ✅ Script can be run directlyFor real-world testing with actual GitHub API access: - -- ✅ Command line arguments work - -- ✅ Help text is displayed```bash - -- ✅ Report is generated correctly# Set real environment variables - -- ✅ Output format is correctexport INPUT_GITHUB_TOKEN="your-github-token" - -export INPUT_ORGANIZATION="QuantEcon" - -## CI Integrationexport INPUT_OUTPUT_FORMAT="markdown" - - - -The GitHub Actions workflow automatically runs:# Run the action - -1. `test-basic.sh` - Validates script structure./generate-report.sh - -2. Additional validation checks for syntax and formatting``` - - - -## OutputNote: Real testing requires a valid GitHub token with appropriate permissions to read organization repositories and issues. - -All tests generate: -- Console output showing test progress -- Success/failure indicators -- Generated `report.md` (when using real GitHub API) +The GitHub Actions workflow (`.github/workflows/ci.yml`) automatically runs: +1. Shell script syntax validation (`bash -n generate-report.sh`) +2. `test-basic.sh` for structure checks +3. Live report generation (on push to main) +4. Dry-run test with invalid token ## Requirements @@ -318,25 +72,15 @@ All tests generate: - **date** (for date calculations) - **GitHub Token** (only for real API calls) -## Notes - -- Tests run from the project root directory -- No token needed for basic validation tests -- Real API tests require a GitHub token with `repo` and `read:org` scopes -- Generated reports are saved to `report.md` -- Tests are designed to be simple and focused - ## Troubleshooting ### "command not found" -Make sure scripts are executable: ```bash chmod +x tests/*.sh chmod +x generate-report.sh ``` ### "GITHUB_TOKEN is required" -Either set environment variable or use command line: ```bash export GITHUB_TOKEN=ghp_xxxxx # or @@ -345,12 +89,9 @@ export GITHUB_TOKEN=ghp_xxxxx ### Date format errors Use ISO format: `YYYY-MM-DD` -```bash -./generate-report.sh --start=2025-10-16 --end=2025-10-18 -``` ## See Also -- [`../README.md`](../README.md) - Main project documentation -- [`../docs/testing.md`](../docs/testing.md) - Comprehensive testing guide -- [`../docs/improvements.md`](../docs/improvements.md) - Technical details +- [../README.md](../README.md) — Main documentation with usage examples +- [../docs/testing.md](../docs/testing.md) — Comprehensive testing guide +- [../docs/improvements.md](../docs/improvements.md) — Technical details From 98818da3962c108075fda690220f9afd9629f918 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 23 Feb 2026 13:51:55 +1100 Subject: [PATCH 2/3] docs: compress copilot instructions from 411 to 119 lines Remove generic sections (code style, best practices, anti-patterns, common tasks, development workflow, testing) that any AI already knows. Keep project-specific rules: no summary files, .tmp/ staging, gh CLI temp files, release process. Fix duplicate post-release verification block. --- .github/copilot-instructions.md | 355 +++----------------------------- 1 file changed, 32 insertions(+), 323 deletions(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index dfcd487..e50ecfd 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -6,93 +6,33 @@ This is a GitHub Action that generates comprehensive weekly activity reports acr ## Documentation Structure -### Root Level Files (Keep Minimal) -- `README.md` - High-level overview, features, usage examples -- `CHANGELOG.md` - Version history and notable changes -- `CONTRIBUTING.md` - How to contribute -- `LICENSE` - MIT license -- `SECURITY.md` - Security policies - -### Documentation (`docs/`) -All detailed documentation goes here: -- `docs/improvements.md` - Technical implementation details -- `docs/architecture.md` - System design and data flow -- `docs/testing.md` - Testing guide -- `docs/validation.md` - Real-world validation examples -- `docs/releases/` - Release-specific notes - -## Critical Rules - -### 1. ❌ DO NOT Create Summary Files - -**Never create files like:** -- `SUMMARY.md` -- `REVIEW_SUMMARY.md` -- `IMPROVEMENTS.md` (at root) -- `BEFORE_AFTER.md` (at root) -- `QUICK_REFERENCE.md` -- Any other standalone summary documents at root level - -**Instead:** -- Update `README.md` for high-level changes -- Update `CHANGELOG.md` for version history -- Add detailed docs to `docs/` folder - -### 2. ✅ DO Update Existing Documentation - -When making changes: -1. **README.md** - Update features, usage examples, high-level info -2. **CHANGELOG.md** - Add entry under `[Unreleased]` section -3. **docs/** - Add or update detailed technical documentation -4. **docs/releases/** - Add release-specific notes when cutting a version - -### 3. Documentation Organization - ``` root/ -├── README.md # Overview, quick start, features -├── CHANGELOG.md # Version history -├── CONTRIBUTING.md # Contribution guidelines -├── LICENSE # License file -├── SECURITY.md # Security policies +├── README.md # Overview, quick start, features +├── CHANGELOG.md # Version history +├── CONTRIBUTING.md # Contribution guidelines +├── LICENSE # MIT license +├── SECURITY.md # Security policies └── docs/ - ├── improvements.md # Technical improvements details - ├── architecture.md # System design - ├── testing.md # Testing guide - ├── validation.md # Validation examples - └── releases/ - ├── v2.0.0.md # Major release notes - └── v1.1.0.md # Minor release notes + ├── improvements.md # Technical implementation details + ├── testing.md # Testing guide + ├── validation.md # Validation examples + ├── hyperlinks.md # Clickable metrics feature + └── releases/ # Release-specific notes ``` -## Code Style - -### Shell Scripts -- Use `set -e` for error handling -- Add comments for complex logic -- Use descriptive variable names -- Add function documentation -- Include debug output for troubleshooting - -### Documentation -- Use clear headings (H1, H2, H3) -- Include code examples -- Add emoji sparingly for visual hierarchy (✅, ❌, ⚠️) -- Keep tables concise and readable -- Use proper markdown formatting - -### Commit Messages -- Use conventional commits: `feat:`, `fix:`, `docs:`, `test:`, `chore:` -- Be specific and descriptive -- Reference issues when applicable +## Critical Rules -## File Editing Guidelines +### 1. DO NOT Create Summary Files -### ⚠️ Critical: Use `.tmp/` for File Staging +Never create standalone summary/review documents at root level (`SUMMARY.md`, `REVIEW_SUMMARY.md`, `QUICK_REFERENCE.md`, etc.). Instead: +- Update `README.md` for high-level changes +- Update `CHANGELOG.md` under `[Unreleased]` for version history +- Add detailed docs to `docs/` -**Problem:** CLI tools like `gh` are more reliable when given a file as input rather than piped/escaped text. Shell escaping for multi-line content (release notes, issue bodies, PR descriptions) is fragile and error-prone. +### 2. Use `.tmp/` for File Staging -**Solution:** Always use the `create_file` tool to write content to `.tmp/`, then pass the file to CLI tools: +CLI tools like `gh` are more reliable when given a file as input rather than piped/escaped text. Shell escaping for multi-line content (release notes, issue bodies, PR descriptions) is fragile and error-prone. ``` 1. create_file -> .tmp/my-replacement.md (use create_file tool) @@ -107,13 +47,7 @@ root/ - Use `create_file` tool for new files, `replace_string_in_file` for edits - For full file replacements: create in `.tmp/`, then `cp` over the target -## GitHub CLI (`gh`) Usage - -### ⚠️ Critical: Use Temporary Files for Input and Output - -**Problem:** `gh` commands are more reliable when given files as input (e.g., `--notes-file`, `--body-file`) rather than inline text with shell escaping. Output can also clutter the terminal with interactive formatting. - -**Solution:** Always use `.tmp/` files for `gh` CLI input, and redirect output to temporary files: +### 3. GitHub CLI (`gh`) — Use Temporary Files for Input and Output ```bash # ❌ BAD - Output may be cluttered with interactive display @@ -134,277 +68,52 @@ gh release create v2.0.0 \ > /tmp/release-output.txt 2>&1 ``` -**Examples:** -```bash -# List releases cleanly -gh release list > /tmp/releases.txt 2>&1 -cat /tmp/releases.txt - -# Check PR status -gh pr view 123 > /tmp/pr-details.txt 2>&1 -grep "state:" /tmp/pr-details.txt - -# Create release with notes file -echo "Release notes here" > /tmp/notes.md -gh release create v1.0.0 --notes-file /tmp/notes.md > /tmp/result.txt 2>&1 -``` - -**Why:** Prevents interactive prompts, formatting codes, and progress indicators from cluttering output. File-based input avoids shell escaping issues with multi-line content. - -## Development Workflow - -### Making Changes - -1. **Code changes** → Update `generate-report.sh` or related files -2. **Test** → Run `./tests/test-improvements.sh` -3. **Document** → Update relevant docs in `docs/` -4. **Changelog** → Add entry to `CHANGELOG.md` under `[Unreleased]` -5. **README** → Update if features/usage changed - -### Adding Features - -```bash -# 1. Implement feature -# 2. Test it -./tests/test-basic.sh -./generate-report.sh --token=ghp_xxx --start=2025-10-16 --end=2025-10-18 - -# 3. Update documentation -# - docs/improvements.md (technical details) -# - README.md (if user-facing) -# - CHANGELOG.md (under [Unreleased]) - -# 4. Commit -git commit -m "feat: add new feature description" -``` - -### Fixing Bugs - -```bash -# 1. Fix the bug -# 2. Test the fix -./tests/test-basic.sh -./generate-report.sh --token=ghp_xxx - -# 3. Update CHANGELOG.md -# Add under [Unreleased] > ### Fixed - -# 4. Commit -git commit -m "fix: resolve specific bug issue" -``` +### 4. Commit Messages -## Testing - -### Before Committing -```bash -# Basic validation -./tests/test-basic.sh - -# Test with real data (optional, requires token) -export GITHUB_TOKEN="your_token" -./generate-report.sh --org=QuantEcon - -# View generated report -cat weekly-report.md -``` +Use conventional commits: `feat:`, `fix:`, `docs:`, `test:`, `chore:` ## Release Process ### Pre-Release Checklist -**CRITICAL: Always complete BEFORE creating any release** - -1. **Review All Documentation in `docs/`** - ```bash - # Check all docs files are up to date - ls -lh docs/ - - # Review each file: - # - docs/README.md - Index current? - # - docs/improvements.md - Reflects all changes? - # - docs/testing.md - Testing procedures current? - # - docs/validation.md - Validation examples accurate? - # - docs/releases/vX.Y.Z.md - Release notes complete? - ``` - -2. **Verify Documentation Completeness** - - [ ] All new features documented - - [ ] All bug fixes explained - - [ ] CLI changes reflected - - [ ] Code examples up to date - - [ ] Cross-references correct - - [ ] No outdated information - - [ ] No corrupted/duplicate content - -3. **Check CHANGELOG.md** - - [ ] All changes under `[Unreleased]` are accurate - - [ ] Version number is correct - - [ ] Release date is set - - [ ] Migration guide included (if breaking changes) - -4. **Verify README.md** - - [ ] Examples show correct version - - [ ] Features list is complete - - [ ] Usage instructions accurate - - [ ] Links work correctly +1. Review all docs in `docs/` are up to date +2. Verify `CHANGELOG.md` — all changes under `[Unreleased]` are accurate +3. Verify `README.md` — examples, features, usage are current +4. No corrupted/duplicate content in any files ### Creating a Release -**Only after completing Pre-Release Checklist:** - -1. **Update CHANGELOG.md** - - Move `[Unreleased]` items to new version section - - Add release date - - Create new empty `[Unreleased]` section - -2. **Create release notes** - ```bash - # Create docs/releases/vX.Y.Z.md with: - # - What's new - # - Breaking changes - # - Migration guide (if needed) - # - Validation results - ``` - -3. **Commit and tag** +1. Move `[Unreleased]` items in `CHANGELOG.md` to new version section with date +2. Create `docs/releases/vX.Y.Z.md` with release notes +3. Commit, tag, and push: ```bash git add . git commit -m "chore: prepare vX.Y.Z release" git push origin main - git tag -a vX.Y.Z -m "Release vX.Y.Z" git push origin vX.Y.Z ``` -4. **Create GitHub release using gh CLI** +4. Create GitHub release (use concise `vX.Y.Z` for title, details go in body): ```bash - # Create release notes file - cat > /tmp/release-notes.md << 'EOF' - ## Release Notes Here - EOF - - # Create release (redirect to temp file!) # IMPORTANT: Use concise semantic version for title (e.g., "v2.1.2" not "v2.1.2 - Description") gh release create vX.Y.Z \ --title "vX.Y.Z" \ --notes-file /tmp/release-notes.md \ --latest \ > /tmp/release-output.txt 2>&1 - - # Verify success cat /tmp/release-output.txt - gh api repos/QuantEcon/action-weekly-report/releases/latest > /tmp/latest.json - jq -r '.tag_name' /tmp/latest.json ``` - - **Note:** Always use concise semantic version format for release titles (e.g., `v2.1.2`). - Descriptions go in the release notes body, not the title. -5. **Update floating version tag (for v2.x.x releases)** +5. Update floating version tag: ```bash - # Update v2 tag to point to latest v2.x.x release git tag -f v2 vX.Y.Z git push origin v2 --force - - # Verify v2 points to the new version - git show-ref --tags | grep v2 ``` -6. **Post-Release Verification** +6. Verify: ```bash - # Check release exists gh api repos/QuantEcon/action-weekly-report/releases/latest > /tmp/release.json 2>&1 - cat /tmp/release.json | jq -r '.tag_name, .name, .published_at' - - # Verify tag - git tag | grep vX.Y.Z - - # Verify v2 floating tag points to new version + jq -r '.tag_name, .name, .published_at' /tmp/release.json git show-ref --tags | grep v2 ``` - gh api repos/QuantEcon/action-weekly-report/releases/latest > /tmp/release.json 2>&1 - cat /tmp/release.json | jq -r '.tag_name, .name, .published_at' - - # Verify tag - git tag | grep vX.Y.Z - ``` - -## Common Tasks - -### Adding New Metrics - -1. Update `generate-report.sh`: - - Add variable initialization - - Add API call for new metric - - Add to report table - - Add to totals - -2. Update documentation: - - `README.md` - Add to features list - - `CHANGELOG.md` - Add under [Unreleased] > ### Added - - `docs/improvements.md` - Technical details - -3. Test: - - Run `./tests/test-basic.sh` for basic validation - - Run `./generate-report.sh --token=xxx` to verify output - -### Fixing Repository Discovery - -1. Modify filtering logic in `generate-report.sh` -2. Test thoroughly with `test-report-preview.sh` -3. Document in `docs/improvements.md` -4. Update `CHANGELOG.md` - -### Improving Performance - -1. Optimize API calls -2. Add caching if beneficial -3. Document changes in `docs/architecture.md` -4. Update `CHANGELOG.md` under Performance section - -## Best Practices - -### API Usage -- Always handle rate limiting -- Use pagination properly -- Add configurable delays -- Provide clear error messages - -### Error Handling -- Fail fast with `set -e` -- Provide actionable error messages -- Log important steps for debugging -- Gracefully handle missing data - -### User Experience -- Clear output messages -- Progress indicators for long operations -- Helpful error messages with solutions -- Comprehensive logging - -## Anti-Patterns to Avoid - -❌ Creating multiple summary files at root level -❌ Duplicating information across files -❌ Writing code without tests -❌ Making changes without updating CHANGELOG -❌ Using hard-coded values (use variables/inputs) -❌ Ignoring rate limits -❌ Poor error messages - -## Questions? - -When in doubt: -1. Check existing patterns in the codebase -2. Keep root level clean (only essential files) -3. Put detailed docs in `docs/` -4. Update `CHANGELOG.md` for all changes -5. Test before committing - -## Project Goals - -- **Complete Coverage**: Capture ALL repository activity -- **Reliability**: Handle rate limits and errors gracefully -- **Performance**: Efficient API usage -- **Maintainability**: Clean, documented code -- **User-Friendly**: Clear reports and error messages From d52b4736ff4e0c0ad93a03f7391f33d4002a32a4 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 23 Feb 2026 16:40:52 +1100 Subject: [PATCH 3/3] fix: address Copilot review feedback on PR #1 - Relax Bash 4.0+ requirement to just Bash (script works with macOS default 3.2) - Remove exact line counts from CHANGELOG entries to avoid drift --- CHANGELOG.md | 4 ++-- CONTRIBUTING.md | 2 +- tests/README.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ab32f51..8c3e488 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,8 +8,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Fixed -- **Corrupted documentation**: Rewrote `docs/testing.md` (was 1275 lines of duplicated/interleaved content, now clean 222 lines) -- **Corrupted tests README**: Rewrote `tests/README.md` (was 357 lines of duplicated content, now clean 97 lines referencing only existing test files) +- **Corrupted documentation**: Rewrote `docs/testing.md` to remove duplicated/interleaved content and restore a clean testing guide +- **Corrupted tests README**: Rewrote `tests/README.md` to remove duplicated content and ensure it only references existing test files - **CLI error message**: Changed misleading `ERROR: GITHUB_OUTPUT environment variable not set!` to a debug message in CLI mode (GITHUB_OUTPUT is only relevant in GitHub Actions) - **Hyperlinks documentation**: Fixed `docs/hyperlinks.md` overview that incorrectly said "metrics greater than 1" when the actual behavior is "metrics greater than 0" diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7150f36..cf95382 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -20,7 +20,7 @@ git clone https://github.com/YOUR-USERNAME/action-weekly-report.git cd action-weekly-report # Ensure dependencies are available -# Required: bash 4.0+, curl, jq, date +# Required: bash, curl, jq, date brew install jq # macOS # sudo apt-get install jq # Linux diff --git a/tests/README.md b/tests/README.md index a318171..36dc4f8 100644 --- a/tests/README.md +++ b/tests/README.md @@ -66,7 +66,7 @@ The GitHub Actions workflow (`.github/workflows/ci.yml`) automatically runs: ## Requirements -- **Bash** 4.0+ +- **Bash** - **jq** (for JSON parsing) - **curl** (for API calls) - **date** (for date calculations)