Skip to content

feat: Add daily token usage analysis and optimization workflows#24192

Merged
pelikhan merged 2 commits intomainfrom
copilot/add-daily-token-usage-workflows
Apr 3, 2026
Merged

feat: Add daily token usage analysis and optimization workflows#24192
pelikhan merged 2 commits intomainfrom
copilot/add-daily-token-usage-workflows

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 2, 2026

gh-aw-firewall has a two-stage token optimization pipeline that has already identified concrete savings (e.g., 21 unused browser tools → ~10,500 tokens/turn). This PR ports that pattern to gh-aw, which has significantly more workflows and would benefit even more from visibility into token consumption.

New Workflows

Stage 1 — Daily Analyzers (9:00/9:30 AM UTC weekdays)

  • copilot-token-usage-analyzer.md — Fetches last 24h Copilot runs via ./gh-aw logs --engine copilot, downloads firewall-audit-logs artifacts for per-model token-usage.jsonl breakdown, creates a 📊 Copilot Token Usage Report: DATE issue with per-workflow stats (tokens, cost, cache hit rate, turns). Auto-closes prior report.
  • claude-token-usage-analyzer.md — Same for Claude, with Anthropic-specific metrics: cache write rate, read/write amortization ratio, TTL guidance (5-min TTL, 1024-token minimum for caching).

Stage 2 — Optimizers (triggered by workflow_run)

  • copilot-token-optimizer.md — Triggered when Copilot analyzer completes. Finds the most expensive workflow, downloads its agent artifacts, diffs declared tools (frontmatter) vs actually-invoked tools (MCP call logs), and creates a ⚡ Copilot Token Optimization: WORKFLOW issue with diff-style recommendations.
  • claude-token-optimizer.md — Same for Claude, adds Anthropic-specific guidance: output tokens cost 3–5× input, cache writes break even at 1.4× reads (1.25 / (1 - 0.1)), flags prompts below 1024-token cache threshold.

Pipeline

Daily schedule
  → Copilot Analyzer → "📊 Copilot Token Usage Report: DATE"
      └─ workflow_run → Copilot Optimizer → "⚡ Copilot Token Optimization: WORKFLOW"

  → Claude Analyzer → "📊 Claude Token Usage Report: DATE"
      └─ workflow_run → Claude Optimizer → "⚡ Claude Token Optimization: WORKFLOW"

Design Decisions

Decision Rationale
bash + gh CLI for artifact downloads Faster and more reliable than MCP for bulk artifact data
strict: true on optimizers Fails fast on unexpected tool calls
workflow_run trigger (not a second cron) Ensures optimizer always sees fresh analyzer data
close-older-issues: true on analyzers Idempotent — no issue accumulation
skip-if-match on optimizers Caps at one open optimization issue per provider
network: defaults Covers *.blob.core.windows.net for artifact downloads

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw on util.test git rev-�� --show-toplevel util.test /usr/bin/git 73/001/test-inligit 9359112/b060/vetrev-parse .cfg git (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name json' --ignore-path ../../../.pr**/*.json /home/REDACTED/work/gh-aw/gh-aw/.g--ignore-path 64/bin/bash /home/REDACTED/wor/opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /home/REDACTED/wor-unsafeptr=false /home/REDACTED/wor-unreachable=false bash --no�� th .prettierignore --log-level=error /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/blog-auditor.md sh /home/REDACTED/wor/tmp/go-build2609359112/b405/constants.test /home/REDACTED/wor-test.testlogfile=/tmp/go-build2609359112/b405/testlog.txt /home/REDACTED/wor-test.paniconexit0 bash (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --show-toplevel x_amd64/vet /usr/bin/git .js' --ignore-pagit (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha --show-toplevel -tests /usr/bin/git --noprofile (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha te '../../../**/*.json' '!../../-p (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha 0/x64/bin/git x_amd64/vet /usr/bin/git --noprofile .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 679815780/.githugit (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git t.go t_test.go 0/x64/bin/node git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.0/x12345 /usr/bin/git SameOutput731459git /tmp/go-build260rev-parse 9359112/b367/vet--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel resolved$ /usr/bin/git --noprofile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha /tmp/go-build2609359112/b454/scripts.test -importcfg /usr/bin/git -s -w -buildmode=exe git rev-�� --show-toplevel -extld=gcc /usr/bin/git --noprofile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel ache/go/1.25.0/x64/pkg/tool/linuTest User /usr/bin/git j36ZOwlHC 9359112/b231/vetrev-parse ache/go/1.25.0/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git --noprofile stmain.go 0/x64/bin/node git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha --noprofile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha path .prettierignore --log-level=error 2>&1 (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha th .prettierigno-errorsas (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha ere config /usr/bin/git remote.origin.urgit (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha /tmp/shared-actions-test653914388 rev-parse /usr/bin/git --noprofile (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha /repos/actions/github-script/git/ref/tags/v8 --jq r,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,disp--show-toplevel -c=4 flow-12345 -importcfg git init�� /home/REDACTED/work/gh-aw/gh-aw/pkg/sliceutil/sliceutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/sliceutil/sliceutil_test.go /usr/bin/git --noprofile (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha g/stringutil/ansi.go g/stringutil/identifiers.go ache/node/24.14.0/x64/bin/node 001' 001' 86_64/sh 9359112/b423/importcfg t-11�� sistency_GoAndJavaScript3312489673/001/test-empty-frontmatter.md -buildtags (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha licyBlockedUsersCommaSeparatedCompiledOutput4060166977/001 /tmp/go-build2609359112/b252/vet.cfg ache/go/1.25.0/x64/pkg/tool/linux_amd64/compile h ../../../.pretgit HEAD tnet/tools/bash ache/go/1.25.0/x64/pkg/tool/linux_amd64/compile -ato�� 9359112/b433/_pkg_.a -buildtags 9359112/b433=> -errorsas b/gh-aw/pkg/slicrev-parse -nilfunc git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 /home/REDACTED/work/gh-aw/gh-aw/.g--delta-base-offset 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/changeset.lock.yml 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path json' --ignore-p-errorsas (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git te '**/*.cjs' '*git .cfg x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /usr/bin/git l.go l_test.go 64/pkg/tool/linu--verify git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha y-test.md .cfg 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha y-token-usage-wo-errorsas /home/REDACTED/wor-ifaceassert x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha y-token-usage-wo-errorsas /home/REDACTED/wor-ifaceassert x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha celain --ignore--errorsas (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha ../pkg/workflow/-errorsas (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha y-token-usage-workflows (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha y-test.md LsRemoteWithRealGitcustom_branch328950461/001' 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo -nolocalimports -importcfg /tmp/go-build2609359112/b414/importcfg -pack /tmp/go-build2609359112/b414/_testmain.go --no�� th .prettierignore --log-level=error (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo bin/bash (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build2609359112/b396/cli.test /tmp/go-build2609359112/b396/cli.test -test.testlogfile=/tmp/go-build2609359112/b396/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true *.md (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build2609359112/b396/importcfg -pack /tmp/go-build2609359112/b396/_testmain.go --no�� th .prettierigno-errorsas (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Create 4 workflows forming a two-stage token optimization pipeline:

Stage 1 - Daily analyzers:
- copilot-token-usage-analyzer.md: Discovers Copilot workflow runs from
  last 24h, downloads token-usage.jsonl artifacts, creates issue with
  per-workflow statistics (tokens, cost, cache hit rate, turns)
- claude-token-usage-analyzer.md: Same for Claude, adds Anthropic-
  specific cache efficiency analysis (write rate, amortization, TTL)

Stage 2 - Optimizers (triggered by workflow_run):
- copilot-token-optimizer.md: Reads usage report, finds most expensive
  workflow, downloads artifacts, compares declared vs used tools,
  creates optimization issue with specific recommendations
- claude-token-optimizer.md: Same for Claude, adds Anthropic pricing
  nuances (output token cost, cache write break-even analysis)

Key design decisions:
- bash + gh CLI for artifact downloads (fast, reliable)
- strict: true on optimizers for safe output validation
- workflow_run trigger chains optimizer to analyzer (not a second cron)
- close-older-issues: true prevents issue accumulation
- skip-if-match caps at one open optimization issue per provider
- network: defaults for artifact downloads (blob.core.windows.net)

Closes #[issue_number]

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/5bf4555f-10f4-46a0-ac16-24b13fe7e27f

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Add daily token usage analysis and optimization workflows feat: Add daily token usage analysis and optimization workflows Apr 3, 2026
Copilot AI requested a review from lpcox April 3, 2026 00:42
Copilot finished work on behalf of lpcox April 3, 2026 00:42
@pelikhan pelikhan marked this pull request as ready for review April 3, 2026 00:43
Copilot AI review requested due to automatic review settings April 3, 2026 00:43
@pelikhan pelikhan merged commit da624a1 into main Apr 3, 2026
60 of 103 checks passed
@pelikhan pelikhan deleted the copilot/add-daily-token-usage-workflows branch April 3, 2026 00:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a two-stage, daily token-usage visibility + optimization pipeline for both Copilot and Claude workflows in gh-aw, producing daily reporting issues and follow-up optimization issues driven by artifact/tool-usage analysis.

Changes:

  • Introduces daily scheduled token usage analyzers for Copilot and Claude that download recent run logs + token-usage.jsonl artifacts and publish a single rolling report issue.
  • Adds optimizer workflows (triggered by workflow_run) that identify the highest-cost workflow and open a single rolling optimization issue with tool/prompt recommendations.
  • Commits compiled .lock.yml workflow outputs for the new analyzer/optimizer workflows.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
.github/workflows/copilot-token-usage-analyzer.md New daily Copilot usage analyzer workflow definition (downloads logs/artifacts + prompts agent to create report issue).
.github/workflows/copilot-token-usage-analyzer.lock.yml Compiled lock workflow for the Copilot usage analyzer.
.github/workflows/copilot-token-optimizer.md New Copilot optimizer workflow definition (selects top workflow + downloads artifacts + prompts agent to create optimization issue).
.github/workflows/copilot-token-optimizer.lock.yml Compiled lock workflow for the Copilot optimizer.
.github/workflows/claude-token-usage-analyzer.md New daily Claude usage analyzer definition (Claude-specific metrics/reporting guidance).
.github/workflows/claude-token-usage-analyzer.lock.yml Compiled lock workflow for the Claude usage analyzer.
.github/workflows/claude-token-optimizer.md New Claude optimizer definition (includes Anthropic cache efficiency guidance + artifact/tool analysis).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +73 to +79
group_by(.workflowName) |
map({
workflow: .[0].workflowName,
total_tokens: (map(.tokenUsage) | add),
total_cost: (map(.estimatedCost) | add),
run_count: length,
avg_tokens: ((map(.tokenUsage) | add) / length),
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The jq pipeline uses group_by(.workflowName) without first sorting by workflowName. In jq, group_by only groups adjacent items, so with the default gh-aw logs ordering (newest-first), this will produce incorrect per-workflow totals and may select the wrong “top” workflow. Sort first (e.g., sort_by(.workflowName) | group_by(.workflowName)) and consider guarding tokenUsage/estimatedCost with // 0 so nulls don’t break add.

Suggested change
group_by(.workflowName) |
map({
workflow: .[0].workflowName,
total_tokens: (map(.tokenUsage) | add),
total_cost: (map(.estimatedCost) | add),
run_count: length,
avg_tokens: ((map(.tokenUsage) | add) / length),
sort_by(.workflowName) |
group_by(.workflowName) |
map({
workflow: .[0].workflowName,
total_tokens: (map(.tokenUsage // 0) | add),
total_cost: (map(.estimatedCost // 0) | add),
run_count: length,
avg_tokens: ((map(.tokenUsage // 0) | add) / length),

Copilot uses AI. Check for mistakes.
Comment on lines +73 to +79
group_by(.workflowName) |
map({
workflow: .[0].workflowName,
total_tokens: (map(.tokenUsage) | add),
total_cost: (map(.estimatedCost) | add),
run_count: length,
avg_tokens: ((map(.tokenUsage) | add) / length),
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The jq pipeline uses group_by(.workflowName) without first sorting by workflowName. In jq, group_by only groups adjacent items, so with the default gh-aw logs ordering (newest-first), this will produce incorrect per-workflow totals and may select the wrong “top” workflow. Sort first (e.g., sort_by(.workflowName) | group_by(.workflowName)) and consider guarding tokenUsage/estimatedCost with // 0 so nulls don’t break add.

Suggested change
group_by(.workflowName) |
map({
workflow: .[0].workflowName,
total_tokens: (map(.tokenUsage) | add),
total_cost: (map(.estimatedCost) | add),
run_count: length,
avg_tokens: ((map(.tokenUsage) | add) / length),
sort_by(.workflowName) |
group_by(.workflowName) |
map({
workflow: .[0].workflowName,
total_tokens: (map(.tokenUsage // 0) | add),
total_cost: (map(.estimatedCost // 0) | add),
run_count: length,
avg_tokens: ((map(.tokenUsage // 0) | add) / length),

Copilot uses AI. Check for mistakes.
Comment on lines +15 to +18
engine: copilot
features:
copilot-requests: true

Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow is named/positioned as the Claude token usage analyzer, but its frontmatter sets engine: copilot (and enables copilot-requests). That means the run will be classified as a Copilot-engine workflow in gh-aw logs, and its own token usage will count toward Copilot totals (potentially skewing Copilot reports/optimizations). If the intent is a Claude-engine pipeline, change engine (and related features/permissions) to claude; otherwise consider renaming/clarifying to avoid confusion and misattribution.

Copilot uses AI. Check for mistakes.
Comment on lines +21 to +24
engine: copilot
features:
copilot-requests: true

Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow is named/positioned as the Claude token optimizer, but its frontmatter sets engine: copilot (and enables copilot-requests). That means the optimizer itself will be logged as a Copilot-engine run and can skew Copilot usage reports; it also diverges from the intended “Claude” optimization pipeline described in the PR. Consider switching engine to claude (and adjusting features/permissions accordingly) or renaming to make it explicit that Copilot is performing the analysis.

Copilot uses AI. Check for mistakes.
Comment on lines +70 to +74
# Find the most expensive workflow (by total tokens across all its runs)
echo "🔍 Identifying most expensive workflow..."
jq -r '
group_by(.workflowName) |
map({
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script’s goal is to optimize the “most expensive workflow”, but the selection sorts by total_tokens (sort_by(.total_tokens)) instead of total_cost. Token totals can mis-rank workflows when model mix/caching differ; if cost is the optimization target, sort by total_cost (with a deterministic tie-break like total_tokens).

Copilot uses AI. Check for mistakes.
Comment on lines +129 to +133
echo "Workflow source not found at $WORKFLOW_MD, searching..."
find .github/workflows -name "*.md" -exec grep -l "^name: $WORKFLOW_NAME" {} \; 2>/dev/null | head -1 | while read -r f; do
echo "Found: $f"
cp "$f" /tmp/token-optimizer/workflow-source.md
done
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflow source discovery is brittle: it assumes ${WORKFLOW_NAME} maps to a .md filename via lowercasing/spaces→hyphens, then falls back to grepping for ^name: $WORKFLOW_NAME in markdown. Many workflows in this repo don’t define name: in frontmatter (e.g. .github/workflows/agent-performance-analyzer.md), so this often won’t find the source and the optimizer will skip declared-vs-used tool analysis. Consider locating the matching .lock.yml by its name: and deriving the .md path from the lock filename (or use workflowPath if available in logs output).

Suggested change
echo "Workflow source not found at $WORKFLOW_MD, searching..."
find .github/workflows -name "*.md" -exec grep -l "^name: $WORKFLOW_NAME" {} \; 2>/dev/null | head -1 | while read -r f; do
echo "Found: $f"
cp "$f" /tmp/token-optimizer/workflow-source.md
done
echo "Workflow source not found at $WORKFLOW_MD, searching for matching lock file..."
LOCK_FILE=""
while IFS= read -r f; do
if grep -Fxq "name: $WORKFLOW_NAME" "$f" 2>/dev/null; then
LOCK_FILE="$f"
break
fi
done < <(find .github/workflows -name "*.lock.yml" -type f 2>/dev/null)
if [ -n "$LOCK_FILE" ]; then
DERIVED_WORKFLOW_MD="${LOCK_FILE%.lock.yml}.md"
if [ -f "$DERIVED_WORKFLOW_MD" ]; then
echo "Found workflow source via lock file: $DERIVED_WORKFLOW_MD"
cp "$DERIVED_WORKFLOW_MD" /tmp/token-optimizer/workflow-source.md
else
echo "Matching lock file found ($LOCK_FILE) but markdown source missing at $DERIVED_WORKFLOW_MD"
fi
fi
if [ ! -f /tmp/token-optimizer/workflow-source.md ]; then
echo "Falling back to markdown frontmatter search..."
find .github/workflows -name "*.md" -exec grep -lFx "name: $WORKFLOW_NAME" {} \; 2>/dev/null | head -1 | while read -r f; do
echo "Found: $f"
cp "$f" /tmp/token-optimizer/workflow-source.md
done
fi

Copilot uses AI. Check for mistakes.
Comment on lines +70 to +74
# Find the most expensive workflow (by total tokens across all its runs)
echo "🔍 Identifying most expensive workflow..."
jq -r '
group_by(.workflowName) |
map({
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script’s goal is to optimize the “most expensive workflow”, but the selection sorts by total_tokens (sort_by(.total_tokens)) instead of total_cost. Token totals can mis-rank workflows when model mix/caching differ; if cost is the optimization target, sort by total_cost (with a deterministic tie-break like total_tokens).

Copilot uses AI. Check for mistakes.
Comment on lines +150 to +159
WORKFLOW_MD_NAME=$(echo "$WORKFLOW_NAME" | tr '[:upper:]' '[:lower:]' | tr ' ' '-')
WORKFLOW_MD=".github/workflows/${WORKFLOW_MD_NAME}.md"
if [ -f "$WORKFLOW_MD" ]; then
echo "Found workflow source: $WORKFLOW_MD"
cp "$WORKFLOW_MD" /tmp/token-optimizer-claude/workflow-source.md
else
find .github/workflows -name "*.md" -exec grep -l "^name: $WORKFLOW_NAME" {} \; 2>/dev/null | head -1 | while read -r f; do
echo "Found: $f"
cp "$f" /tmp/token-optimizer-claude/workflow-source.md
done
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflow source discovery is brittle: it assumes ${WORKFLOW_NAME} maps to a .md filename via lowercasing/spaces→hyphens, then falls back to grepping for ^name: $WORKFLOW_NAME in markdown. Many workflows in this repo don’t define name: in frontmatter (e.g. .github/workflows/agent-performance-analyzer.md), so this often won’t find the source and the optimizer will skip declared-vs-used tool analysis. Consider locating the matching .lock.yml by its name: and deriving the .md path from the lock filename (or use workflowPath if available in logs output).

Suggested change
WORKFLOW_MD_NAME=$(echo "$WORKFLOW_NAME" | tr '[:upper:]' '[:lower:]' | tr ' ' '-')
WORKFLOW_MD=".github/workflows/${WORKFLOW_MD_NAME}.md"
if [ -f "$WORKFLOW_MD" ]; then
echo "Found workflow source: $WORKFLOW_MD"
cp "$WORKFLOW_MD" /tmp/token-optimizer-claude/workflow-source.md
else
find .github/workflows -name "*.md" -exec grep -l "^name: $WORKFLOW_NAME" {} \; 2>/dev/null | head -1 | while read -r f; do
echo "Found: $f"
cp "$f" /tmp/token-optimizer-claude/workflow-source.md
done
WORKFLOW_MD=""
if [ -n "${WORKFLOW_PATH:-}" ] && [ -f "$WORKFLOW_PATH" ]; then
WORKFLOW_MD="$WORKFLOW_PATH"
else
WORKFLOW_MD_NAME=$(echo "$WORKFLOW_NAME" | tr '[:upper:]' '[:lower:]' | tr ' ' '-')
CANDIDATE_WORKFLOW_MD=".github/workflows/${WORKFLOW_MD_NAME}.md"
if [ -f "$CANDIDATE_WORKFLOW_MD" ]; then
WORKFLOW_MD="$CANDIDATE_WORKFLOW_MD"
else
LOCK_FILE=$(find .github/workflows -name "*.lock.yml" -exec grep -l "^name: $WORKFLOW_NAME$" {} \; 2>/dev/null | head -1)
if [ -n "$LOCK_FILE" ]; then
CANDIDATE_WORKFLOW_MD="${LOCK_FILE%.lock.yml}.md"
if [ -f "$CANDIDATE_WORKFLOW_MD" ]; then
WORKFLOW_MD="$CANDIDATE_WORKFLOW_MD"
fi
fi
if [ -z "$WORKFLOW_MD" ]; then
WORKFLOW_MD=$(find .github/workflows -name "*.md" -exec grep -l "^name: $WORKFLOW_NAME$" {} \; 2>/dev/null | head -1)
fi
fi
fi
if [ -n "$WORKFLOW_MD" ] && [ -f "$WORKFLOW_MD" ]; then
echo "Found workflow source: $WORKFLOW_MD"
cp "$WORKFLOW_MD" /tmp/token-optimizer-claude/workflow-source.md
else
echo "Workflow source not found for: $WORKFLOW_NAME"

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Add daily token usage analysis and optimization workflows

4 participants