perf: optimize doc-maintainer token usage (~27% reduction)#1645
Merged
Conversation
Implements recommendations from token optimization issue #1643: 1. Remove unused github: toolset — the agent uses bash for all git/file ops and safeoutputs for PR creation. The 22 GitHub MCP tools were loaded but never called, adding ~13,200 tokens/turn to the system prompt. Estimated savings: ~422K tokens/run (~14%). 2. Pre-compute git changes and doc file list in steps: block — eliminates the agent's discovery phase (3-5 turns of running git log, find). Step outputs are injected into the prompt via template variables. Estimated savings: ~190K tokens/run (~8%). 3. max-turns guard — not implemented because the copilot engine does not support this feature. The existing timeout-minutes: 15 serves as the cost cap. Expected impact: ~$3.91 → ~$2.55/run (-35% cost), 3,003K → ~2,200K tokens/run (-27%), 22 → 0 GitHub tools loaded (-100%). Closes #1643 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
Contributor
Smoke Test Results
Overall: PASS
|
Contributor
Smoke Test ResultsPR Titles:
|
Contributor
🤖 Smoke Test ResultsPR: perf: optimize doc-maintainer token usage (~27% reduction) — @lpcox
Overall: PASS
|
Contributor
Smoke Test: GitHub Actions Services Connectivity ✅
All checks passed.
|
Contributor
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
doc-maintainerworkflow costs ~$3.91/run and uses ~3,003K tokens. Analysis from the token optimization advisor (#1643) identified that:bashfor all git/file operationsgit log,find) that can be pre-computedChanges
1. Remove unused
github:toolset (−14% tokens)The agent never calls any GitHub MCP tools. It uses
bash(git/find/cat) for discovery andsafeoutputs.create_pull_requestfor the PR. Each tool schema adds ~600 tokens to every turn's system prompt.Before: 22 GitHub MCP tools loaded → ~13,200 tokens/turn × 32 turns = ~422K tokens/run
After: 0 GitHub MCP tools loaded
2. Pre-compute git changes in
steps:block (−8% tokens)Added two pre-agent steps:
git-changes: Runsgit log --since="7 days ago" --name-onlyand injects results into the prompt via${{ steps.git-changes.outputs.RECENT_COMMITS }}doc-files: Lists all docs viafind docs/ -name "*.md"and injects via${{ steps.doc-files.outputs.DOC_FILES }}This eliminates the agent's discovery phase and lets it jump straight to analyzing diffs.
3.
max-turnsguard (not implemented)The copilot engine does not support
max-turns. The existingtimeout-minutes: 15serves as the cost guard.Expected Impact
Closes #1643