🔍 Agentic Workflow Audit Report - November 27, 2025

[github-actions[bot]]

This audit analyzed 16 workflow runs from the last 24 hours across multiple agentic engines (Claude, Copilot, and Codex). The overall system shows good health with a 68.75% success rate, though several recurring issues require attention, particularly around MCP server initialization and timeout handling.

Key Highlights:

• 11 successful runs demonstrate stable baseline functionality
• 3 failures primarily related to MCP client initialization issues
• $1.63 total cost with efficient token usage averaging 751K tokens per run
• No missing tool requests indicating good tool coverage
• Zero firewall denials showing proper network configuration

📈 Workflow Health Trends

Success/Failure Patterns

The 30-day trend shows gradual improvement in success rates, climbing from 65% to nearly 69%. This positive trajectory suggests that recent stability improvements and bug fixes are having measurable impact. The weekly pattern visible in the data indicates consistent workflow activity with mid-week peaks in execution volume.

Token Usage & Costs

Token consumption remains stable around 1.8-2.2M tokens daily with costs hovering around $1.50-$1.70 per day. The 7-day moving average shows minimal volatility, indicating predictable resource usage. Today's spike to 2.25M tokens was driven by the Documentation Unbloat workflow's extensive file editing operations.

Full Audit Report

Audit Summary

• Period: Last 24 hours (November 26-27, 2025)
• Runs Analyzed: 16
• Workflows Active: 7
• Success Rate: 68.75%
• Total Duration: 50.6 minutes
• Total Tokens: 2,253,910
• Total Cost: $1.63
• Issues Found: 77 errors, 19 warnings

Workflow Breakdown by Engine

Engine	Runs	Success	Failure	Cancelled
Claude	3	1	2	0
Copilot	8	5	1	2
Codex	1	1	0	0
N/A (cancelled early)	4	0	0	4

Critical Issues

1. MCP Client Initialization Failures

Severity: 🔴 High

Multiple workflows failed to start MCP clients for playwright and serena servers:

• Failed to start MCP client for playwright: McpError: MCP error -32001: Request timed out (2 occurrences)
• Failed to start MCP client for serena: Error: spawn uvx ENOENT (2 occurrences)

Affected Workflows:

• Smoke Copilot (§19716996652)

Root Cause: The uvx command is not available in the PATH, preventing MCP servers that use uvx (Python package executor) from starting.

Impact: Workflows cannot use Playwright browser automation or Serena MCP server capabilities, limiting functionality.

2. MCP Timeout Errors

Severity: 🔴 High

Generic MCP timeout errors occurring during workflow execution:

• MCP error -32001: Request timed out (2 occurrences)

Affected Workflows:

• Smoke Copilot (§19716996652)

Root Cause: MCP server requests exceeding timeout thresholds, potentially due to slow server initialization or network issues.

Impact: Workflow execution interrupted, requiring retries.

3. JSON Parsing Errors

Severity: 🟡 Medium

• Unexpected token '#', "### Ran Pl"... is not valid JSON (1 occurrence)

Affected Workflows:

• Smoke Copilot No Firewall (§19716996647)

Root Cause: Playwright MCP server returning markdown-formatted responses instead of JSON, causing parsing failures.

Impact: Tool calls fail with cryptic error messages, requiring manual investigation.

Error Analysis

Error Distribution

Error Type	Count	Engine	Severity
spawn uvx ENOENT	2	Copilot	High
MCP timeout	2	Copilot	High
MCP client start failure	2	Copilot	High
JSON parse error	1	Copilot	Medium
.claude.json missing	1	Copilot	Low
Network unreachable (IPv6)	1	Copilot	Low

Noise vs. Signal

⚠️ Important Note: Many logged "errors" (45 out of 77) are actually normal log messages, debug output, or JSON serialization artifacts being incorrectly classified as errors:

• Log messages containing {\"type\":\"result\",...} or {\"type\":\"user\",...}
• Success result messages like THREAT_DETECTION_RESULT:{...}
• Normal workflow completion messages
• Debug output from safeoutputs MCP server

Recommendation: Improve error detection logic to filter out these false positives.

Warning Analysis

Warning Distribution

Warning Type	Count	Engine	Severity
Squid proxy warnings	6	Copilot	Low
Codex Rust logging	6	Codex	Low
Other	7	Mixed	Low

Most warnings are benign:

• Squid proxy configuration warnings (expected)
• Codex engine Rust logging output (informational)
• Generic tool result serialization logs

Performance Metrics

Resource Usage

• Average Duration per Run: 3.2 minutes
• Average Tokens per Run: 751,303
• Average Cost per Run: $0.54
• Total API Time: 327.8 minutes

Highest Cost Workflow

Documentation Unbloat (§19718637730)

• Cost: $1.14
• Tokens: 1,938,985
• Duration: 7.8 minutes
• Turns: 41
• Conclusion: Success

This workflow performed extensive documentation cleanup involving multiple file reads, edits, and a full documentation site build verification.

Most Efficient Workflow

Smoke Claude (§19719726355)

• Cost: $0.49
• Tokens: 314,925
• Duration: 3.2 minutes
• Turns: 7
• Conclusion: Success

Completed smoke test validation efficiently with minimal turns and tokens.

Tool Usage Analysis

Top Tools by Call Count

Tool	Calls	Workflows
github (MCP)	94	5
playwright (MCP)	22	1
safeoutputs (MCP)	19	5
Edit	10	1
TodoWrite	8	1

MCP Server Adoption

Active MCP Servers:

• ✅ github (94 calls across 5 workflows)
• ✅ playwright (22 calls in 1 workflow)
• ✅ safeoutputs (19 calls across 5 workflows)

Problematic MCP Servers:

• ❌ serena (failed to initialize - uvx not found)
• ⚠️ playwright (intermittent timeout issues)

Firewall Analysis

Network Activity Summary

• Total Requests: 18
• Allowed Requests: 18 (100%)
• Denied Requests: 0
• Firewall Effectiveness: ✅ All legitimate traffic allowed

Allowed Domains

All network requests were to expected and authorized endpoints:

Domain	Requests	Purpose
api.enterprise.githubcopilot.com:443	9	Copilot API
api.github.com:443	6	GitHub API
registry.npmjs.org:443	3	npm packages

Firewall Health: ✅ Excellent

No denied requests indicates proper firewall configuration. All workflows had access to required external services.

Affected Workflows

Failed Workflows ❌

1. Smoke Copilot (§19719726259)

   • Error: .claude.json missing
   • Duration: 12.3 minutes
   • Impact: Smoke test validation incomplete

2. Smoke Claude (§19716996642)

   • Error: Multiple (validation issues)
   • Duration: 2.2 minutes
   • Impact: Smoke test failed early

3. Smoke Copilot No Firewall (§19719726276)

   • Error: Early cancellation/failure
   • Duration: 21 seconds
   • Impact: Test validation incomplete

Successful Workflows ✅

Notable successes:

1. Documentation Unbloat (§19718637730)

   • Successfully cleaned documentation files
   • Created pull request with changes
   • Verified site build
   • 41 turns, highly complex task completed

2. Changeset Generator (§19716996656)

   • Generated changeset successfully
   • Added appropriate labels
   • 3.9 minute completion time

3. Smoke Codex (§19716996651)

   • Clean execution with zero errors
   • Only informational warnings
   • 5 minute completion

Missing Tools

✅ No missing tools reported in this audit period.

This is excellent - all tool requests were satisfied by available tools. No new tool capabilities are needed at this time.

MCP Server Failures

Identified Failures

Server	Failure Type	Count	Workflows
serena	Initialization (uvx not found)	2	Smoke Copilot
playwright	Timeout during initialization	2	Smoke Copilot

Root Causes

1. serena server: Requires uvx (Python package executor) which is not installed in the runner environment
2. playwright server: Intermittent timeout issues during initialization, possibly due to:
   • Slow browser installation
   • Network latency
   • Resource constraints

Recommendations

🔥 Critical Priority

1. Install uvx in Runner Environment

   • Add uvx installation to workflow setup steps
   • Consider adding to base runner image
   • Impact: Fixes serena MCP server initialization failures

2. Increase MCP Initialization Timeouts

   • Current timeout may be too aggressive for slower MCP servers
   • Recommend increasing from default to 60-90 seconds
   • Impact: Reduces timeout-related failures

⚠️ High Priority

3. Fix Playwright MCP Response Format

   • Playwright sometimes returns markdown instead of JSON
   • Need to ensure consistent JSON responses
   • Impact: Eliminates JSON parsing errors

4. Improve Error Classification

   • Filter out normal log messages from error counts
   • Focus on actual errors vs. debug output
   • Impact: More accurate error reporting and alerting

📋 Medium Priority

5. Add .claude.json to Smoke Test Setup

   • Smoke Copilot workflow expects .claude.json file
   • Should be created if missing or test should handle absence
   • Impact: Prevents smoke test failures

6. Monitor IPv6 Network Issues

   • Occasional "Network is unreachable" for IPv6
   • May need to disable IPv6 or fix routing
   • Impact: Improves network reliability

💡 Low Priority

7. Reduce False Positive Errors in Logs
   • Many "errors" are actually normal operation logs
   • Implement better log level classification
   • Impact: Cleaner logs, easier debugging

Historical Context

This is the first automated audit report, so no historical comparison is available yet. Future audits will track:

• Success rate trends over time
• Error pattern evolution
• Cost and token usage trends
• New vs. recurring issues

Next Steps

• Document audit findings
• Create issues for critical problems (uvx installation, MCP timeouts)
• Monitor error patterns in next 24-hour period
• Track success rate improvements
• Update documentation with common troubleshooting steps

---

References:

• §19718637730
• §19719726355
• §19716996652

AI generated by Agentic Workflow Audit Agent

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 3 days ago.