🔍 Agentic Workflow Audit Report - November 22, 2025

[github-actions[bot]]

🔍 Agentic Workflow Audit Report - November 22, 2025

This audit analyzed 88 workflow runs from the last 24 hours across the gh-aw repository to identify issues, patterns, and opportunities for improvement.

Executive Summary

The workflow ecosystem shows healthy activity with a 72.7% success rate across 88 runs. Three AI engines (Claude, Copilot, Codex) processed nearly 28 million tokens at an estimated cost of $20.15. One missing tool issue was identified affecting the Smoke Copilot workflow, and 15 runs failed requiring investigation.

Key highlights:

• 64 successful runs demonstrating solid workflow reliability
• 15 failed runs concentrated in Security Fix PR, Changeset Generator, and Release Highlights workflows
• 1 missing tool (Playwright) requested 5 times by Smoke Copilot workflow
• Average of 315K tokens per run indicating complex agentic tasks
• No MCP server failures detected

📈 Workflow Health Trends

Success/Failure Patterns

The workflow ecosystem maintained a 72.7% success rate over the last 24 hours. The majority of runs completed successfully (64 runs), with 15 failures and 7 cancellations. The cancellations appear to be intentional (e.g., Tidy workflow cancelled when no changes needed), rather than system failures.

Token Usage & Costs

Resource consumption totaled 27.7M tokens at an estimated cost of $20.15 for the 24-hour period. The high token count reflects the complexity of agentic workflows, with individual runs averaging 315K tokens - indicating deep code analysis, research, and multi-turn agent interactions.

Full Report Details

Audit Summary

• Period: Last 24 hours (November 21-22, 2025)
• Runs Analyzed: 88
• Workflows Active: 43 distinct workflows
• Success Rate: 72.7%
• Issues Found: 1 (missing tool)

Run Distribution by Conclusion

Status	Count	Percentage
✅ Success	64	72.7%
❌ Failure	15	17.0%
🚫 Cancelled	7	8.0%
⏳ In Progress	2	2.3%

Engine Distribution

Engine	Runs	Percentage
Claude	40	45.5%
Copilot	39	44.3%
Codex	9	10.2%

Missing Tools

One missing tool was detected during the audit period:

Tool Name	Request Count	Workflows Affected	Reason
playwright	5	Smoke Copilot	Browser automation testing tool not available in workflow environment

Impact: The Smoke Copilot workflow attempted to use Playwright for browser automation tests but the tool is not currently available in the agentic workflow environment.

Affected Runs:

• §19559090177
• §19559140903
• §19561625043
• §19579261767
• §19586983136

Error and Warning Analysis

Error Patterns Detected

Pattern ID	Count	Description
common-generic-error	256	General error messages from workflow logs
common-gh-actions-error	1	GitHub Actions workflow command errors

Warning Patterns Detected

Pattern ID	Count	Description
common-generic-warning	244	General warning messages
copilot-permission-denied	94	Permission denied errors in Copilot workflows
codex-rust-warning	37	Rust compilation warnings in Codex workflows
copilot-unauthorized	10	Unauthorized access attempts
copilot-forbidden	7	Forbidden access attempts
common-gh-actions-warning	1	GitHub Actions workflow warnings

Notable: 94 permission-denied warnings suggest Copilot workflows may be attempting operations requiring additional permissions.

MCP Server Failures

✅ No MCP server failures detected during the audit period. All configured MCP servers (GitHub, gh-aw, safeoutputs) operated without interruption.

Performance Metrics

• Total Token Usage: 27,731,707 tokens
• Total Cost (24h): $20.15
• Average Tokens per Run: 315,133 tokens
• Total Agent Turns: 827
• Average Turns per Run: 9.4

High Token Usage Runs

Top 5 workflows by token consumption:

Workflow	Run ID	Tokens	Cost	Engine
Security Fix PR	§19548455209	3,509,905	$1.70	Claude
Daily Documentation Updater	§19561671925	1,979,216	$1.22	Claude
Security Fix PR	§19585797303	1,401,535	$0.86	Claude
Copilot Agent PR Analysis	§19576111301	1,226,932	$0.92	Claude
Developer Documentation Consolidator	§19561824647	1,193,698	$0.73	Claude

Failed Runs

15 workflow runs failed during the audit period. Key patterns:

Security Fix PR (4 failures)

• Multiple failures attempting to fix security alerts
• High token consumption (averaging 1.7M tokens per attempt)
• Runs: §19587010229, §19585797303, §19585466104, §19548455209

Changeset Generator (3 failures)

• Failures with 0 tokens consumed suggest early termination
• Runs: §19586088247, §19585195318, §19584497219

Release Highlights Generator (2 failures)

• Both failures with 0 tokens consumed
• Runs: §19585051679, §19583350860

Duplicate Code Detector (2 failures)

• Codex engine failures
• Runs: §19583374500, §19551191192

Other Failures

• Daily Documentation Updater: §19561671925 (high token usage before failure)
• Smoke Codex: §19586966195
• Tidy: §19576235076
• Lockfile Statistics: §19565335869

Affected Workflows

Most Active Workflows (by run count)

Workflow	Runs	Success	Failed	Cancelled
Go Pattern Detector	16	16	0	0
Tidy	16	12	1	3
Changeset Generator	7	3	3	1
Smoke Claude	6	6	0	0
Smoke Codex	6	5	1	0
Smoke Copilot	6	6	0	0
Security Fix PR	4	0	4	0
Duplicate Code Detector	3	1	2	0

Recommendations

1. High Priority: Security Fix PR Workflow

Issue: 100% failure rate (4/4 attempts failed) with high resource consumption

Recommendation:

• Investigate why Security Fix PR workflow consistently fails after consuming significant tokens (avg 1.7M tokens)
• Review recent changes to the workflow or security fix strategies
• Consider adding timeout or complexity limits to prevent expensive failed runs
• Examine the specific security alerts being addressed for patterns

2. Medium Priority: Add Playwright Support

Issue: Smoke Copilot workflow requesting missing Playwright tool (5 occurrences)

Recommendation:

• Evaluate whether Playwright is necessary for Smoke Copilot workflow
• If needed, add Playwright to the workflow environment via setup action or MCP server
• If not needed, update workflow to use alternative testing approach or remove Playwright dependency

3. Medium Priority: Investigate Early Termination Failures

Issue: Changeset Generator and Release Highlights workflows failing with 0 tokens consumed

Recommendation:

• These failures suggest issues in workflow activation or pre-flight checks
• Review workflow trigger conditions and activation logic
• Check for recent changes to these workflows that might cause early exits

4. Low Priority: Address Permission Warnings

Issue: 94 permission-denied warnings in Copilot workflows

Recommendation:

• Audit Copilot workflow permissions to identify restricted operations
• Determine if additional GitHub token scopes are needed
• Document any intentional permission restrictions to reduce noise

5. Low Priority: Monitor Token Consumption

Issue: High average token usage (315K per run) impacts cost

Recommendation:

• Current spending rate is ~$20/day
• Monitor for unexpected increases
• Consider implementing token budgets for individual workflows
• Review high-consumption workflows (Security Fix PR, Documentation tasks) for optimization opportunities

Historical Context

This is the first automated audit using the new Agentic Workflow Audit Agent. Historical data will be accumulated over time to identify trends and patterns across multiple days.

Next Steps

• ✅ Investigation of Security Fix PR workflow failures
• ✅ Decision on Playwright tool support for Smoke Copilot
• ✅ Review early termination patterns in Changeset Generator
• ⏳ Monitor success rate trends over next 7 days
• ⏳ Establish baseline metrics for comparison

---

References:

• §19587010229 - Security Fix PR failure (high cost)
• §19561671925 - Daily Documentation Updater failure (high cost)
• §19548455209 - Security Fix PR failure (highest token usage)

AI generated by Agentic Workflow Audit Agent

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 1 week ago.