feat: Add daily token usage analysis and optimization workflows

[lpcox]

Summary

The gh-aw-firewall repo has implemented a set of agentic workflows that automatically track token usage across all agentic workflows and generate actionable optimization recommendations. These have already identified concrete savings (e.g., excluding 21 unused browser tools saving ~10,500 tokens/turn). This proposal is to create similar workflows for gh-aw.

What exists in gh-aw-firewall

Four workflows form a two-stage pipeline per AI provider:

Stage 1: Token Usage Analyzers (daily)

• Copilot Token Usage Analyzer — Discovers completed Copilot-engine workflow runs from the past 24 hours, downloads token-usage.jsonl artifacts, and creates an issue with per-workflow statistics (total/billable tokens, input/output ratio, cache hit rate, request count, estimated cost, historical trends)
• Claude Token Usage Analyzer — Same for Claude/Anthropic workflows, with provider-specific metrics (cache write rate, cache write amortization, Anthropic pricing tiers)

Stage 2: Token Optimizers (triggered by analyzers)

• Copilot Token Optimizer — Reads the latest usage report, identifies the most expensive workflow, downloads its artifacts to compare declared vs. actually-used tools, analyzes the workflow .md source, and creates an optimization issue with specific recommendations and estimated token savings
• Claude Token Optimizer — Same for Claude, with Anthropic-specific insights (cache write cost analysis, TTL considerations)

How they chain together

Daily schedule
  → Copilot Analyzer runs → creates "📊 Copilot Token Usage Report" issue
    → Copilot Optimizer triggers (workflow_run) → creates "⚡ Copilot Token Optimization" issue

  → Claude Analyzer runs → creates "📊 Claude Token Usage Report" issue  
    → Claude Optimizer triggers (workflow_run) → creates "⚡ Claude Token Optimization" issue

Each run auto-closes previous report/optimization issues via close-older-issues: true.

Key design decisions

• bash + gh CLI preferred over MCP for artifact downloads (faster, more reliable for bulk data)
• strict: true on optimizers for safe output validation
• workflow_run trigger chains optimizer to analyzer completion (not a second cron)
• Provider-scoped analysis — each analyzer only looks at workflows using its provider, avoiding scope creep
• Progressive disclosure — detailed per-workflow metrics wrapped in <details> blocks
• Idempotent — close-older-issues prevents issue accumulation; optimizers use skip-if-match to cap at one open optimization issue per provider

Proposal for gh-aw

gh-aw has significantly more agentic workflows than gh-aw-firewall, making token visibility even more valuable. Proposed approach:

1. Create Copilot and Claude token usage analyzers adapted for gh-aw's workflow inventory
2. Create Copilot and Claude token optimizers that analyze the most expensive workflow and recommend specific changes
3. Leverage existing token-usage.jsonl artifacts already produced by gh-aw runs
4. Use gh-aw's own shared components (shared/mcp-pagination.md, shared/reporting.md) where applicable

What would need to change vs. the firewall versions

• Workflow discovery — gh-aw has many more workflows to scan; the analyzer prompt would need an updated list or dynamic discovery
• Post-processing — gh-aw-firewall has a custom post-processor for --excluded-tools injection; gh-aw may want a different mechanism for applying tool exclusions
• Pricing models — gh-aw may use different model tiers; pricing tables should be configurable
• Network allowlist — *.blob.core.windows.net needed for artifact downloads

Reference implementations

• Copilot Token Usage Analyzer
• Claude Token Usage Analyzer
• Copilot Token Optimizer
• Claude Token Optimizer
• Example optimization PR: github/gh-aw-firewall#1625 — excluded 21 unused browser tools, saving ~10,500 tokens/turn

[lpcox]

@pelikhan take a look