fix: align token workflows with gh-aw logs --json schema and add shared log caching#24395
Merged
fix: align token workflows with gh-aw logs --json schema and add shared log caching#24395
Conversation
gh aw logs --json returns an object with a .runs array (not a bare array), and run objects use snake_case field names. All 4 token workflows (copilot/claude analyzers and optimizers) assumed a bare array with camelCase fields, causing jq errors like: Cannot index array with string "workflowName" Changes: - Extract .runs array from JSON object before processing - Replace camelCase fields with snake_case (workflow_name, token_usage, database_id, created_at) - Replace non-existent estimatedCost with 0 placeholder - Update documentation sections with correct field names Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes the token analyzer/optimizer workflows by updating their gh aw logs --json parsing to match the actual output schema (top-level object with a .runs array and snake_case fields), preventing jq indexing failures during workflow execution.
Changes:
- Write
gh aw logs --jsonoutput to a raw file and extract.runsinto the array file consumed by subsequent jq steps. - Update jq field references from camelCase to snake_case (e.g.,
databaseId→database_id,workflowName→workflow_name). - Recompile the corresponding
.lock.ymlworkflow files.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/copilot-token-usage-analyzer.md | Extracts .runs and updates run field names in analyzer + embedded docs |
| .github/workflows/copilot-token-usage-analyzer.lock.yml | Recompiled lock with the updated download/parsing logic |
| .github/workflows/copilot-token-optimizer.md | Extracts .runs and updates optimizer jq aggregation to snake_case fields |
| .github/workflows/copilot-token-optimizer.lock.yml | Recompiled lock with the updated optimizer logic |
| .github/workflows/claude-token-usage-analyzer.md | Extracts .runs and updates run field names in analyzer + embedded docs |
| .github/workflows/claude-token-usage-analyzer.lock.yml | Recompiled lock with the updated download/parsing logic |
| .github/workflows/claude-token-optimizer.md | Extracts .runs and updates optimizer jq aggregation to snake_case fields |
| .github/workflows/claude-token-optimizer.lock.yml | Recompiled lock with the updated optimizer logic |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 8/8 changed files
- Comments generated: 2
Collaborator
|
@copilot Extract a shared agentic workflow that downloads logs and stores them in a common cache to avoid rate-limiting when running these workflows. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Recompile copilot-token-optimizer and claude-token-optimizer lock files to match the sort_by(.workflow_name) additions made via UI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
|
@lpcox wait for copilot. We need to refactor the logs download. It's killing our gh api usage |
…d rate-limiting Agent-Logs-Url: https://github.com/github/gh-aw/sessions/37b38e9c-5938-4c6f-a082-9bc64b2a8b7b Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…into fix/token-workflow-json-schema # Conflicts: # .github/workflows/claude-token-optimizer.lock.yml # .github/workflows/copilot-token-optimizer.lock.yml Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
Added a new
All 4 token workflows now check for the cached artifact first ( Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Copilot
AI
changed the title
fix: align token workflows with gh-aw logs --json schema
fix: align token workflows with gh-aw logs --json schema and add shared log caching
Apr 3, 2026
Collaborator
|
@lpcox ready |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Copilot Token Optimizer workflow failed in run 23962678922 with:
All 4 token workflows (copilot/claude analyzers and optimizers) assumed
gh aw logs --jsonoutputs a bare JSON array with camelCase field names. In reality, it outputs an object with a.runsarray, and run objects use snake_case field names.Additionally, each workflow independently called
gh aw logs --json, causing redundant API calls and potential rate-limiting when all 4 workflows run in sequence each day.Fix
.runsarray from the JSON object before processingworkflowName→workflow_nametokenUsage→token_usagedatabaseId→database_idcreatedAt→created_atestimatedCostwith0placeholdersort_by(.workflow_name)beforegroup_by(.workflow_name)in optimizers for deterministic groupingtoken-logs-fetch.mdshared agentic workflow that pre-fetches both Copilot and Claude logs once daily (~08:45) and caches them viacache-memorygh aw logs --jsonif no valid cache exists)Files Changed
.github/workflows/token-logs-fetch.md(new).github/workflows/copilot-token-optimizer.md.github/workflows/claude-token-optimizer.md.github/workflows/copilot-token-usage-analyzer.md.github/workflows/claude-token-usage-analyzer.md.lock.ymlfiles (recompiled)Testing
make fmt && make lintpass with 0 issues