[prompt-clustering] Copilot Agent Prompt Clustering Analysis — 904 Tasks, 8 Clusters

[github-actions[bot]]

Summary

Analysis of 904 copilot agent task prompts from the github/gh-aw repository (dataset window: January 21 – February 7, 2026). TF-IDF vectorization with cosine-similarity K-means (k=8) was used to cluster the prompts.

Metric	Value
Total tasks analyzed	904
Merged (success)	623 (68.9%)
Closed (not merged)	277 (30.6%)
Open / in-flight	4 (0.4%)
Clusters identified	8
Best success-rate cluster	Agentic Workflow Maintenance (81.6%)
Lowest success-rate cluster	MCP & CLI Tooling (57.0%)

---

Cluster Overview

#	Theme	Tasks	Merged	Success	Avg Commits	Avg Files
1	MCP & CLI Tooling	207	118	57.0%	3.9	29.2
2	Code Quality & Testing	151	102	67.5%	3.4	13.0
3	Agentic Workflow Maintenance	114	93	81.6%	4.4	18.5
4	Safe-Outputs & Project Infrastructure	102	78	76.5%	4.7	20.0
5	Workflow Failure Investigation	100	78	78.0%	3.0	8.1
6	Investigation & Debugging	94	68	72.3%	3.3	26.2
7	Campaign & Security Automation	87	49	56.3%	4.3	8.7
8	CI Job Analysis & Go/TypeScript Fixes	49	37	75.5%	3.4	22.8

---

Cluster Details

Cluster 1: MCP & CLI Tooling

207 tasks (22.9%) | Success: 57.0% | Avg commits: 3.9 | Avg files: 29.2

Top keywords: mcp, command, version, copilot, github, cli, server

Tasks related to MCP server upgrades (Sentry, gh-aw-mcpg, etc.), CLI command additions, and Copilot agent configuration. The largest cluster (22.9% of all tasks) but lowest success rate (57%), suggesting MCP/version-update tasks are frequently iterated or abandoned mid-stream.

Representative PRs:

• ✅ #11050 — chore: Update Sentry MCP server to 0.27.0
• ✅ #11064 — Add interactive engine selection and secret configuration to init command
• ✅ #11067 — Add missing get_repository tool to repos toolset

---

Cluster 2: Code Quality & Testing

151 tasks (16.7%) | Success: 67.5% | Avg commits: 3.4 | Avg files: 13.0

Top keywords: code, test, files, quality, validation, schema, lines

Tasks fixing code validation issues, TypeScript type errors, schema mismatches, and test regressions. High volume (16.7%) with moderate success (67.5%) — well-understood bug-fix patterns that often require multiple iterations.

Representative PRs:

• ✅ #11058 — Fix ephemerals tests after blockquote prefix requirement in PR Fix expiration detection for quoted footers and legacy format #11036
• ✅ #11069 — Fix TypeScript type error in close_older_issues.cjs - add type guard
• ✅ #11071 — Remove active/passive campaign distinction

---

Cluster 3: Agentic Workflow Maintenance

114 tasks (12.6%) | Success: 81.6% | Avg commits: 4.4 | Avg files: 18.5

Top keywords: agentic, agentic workflow, agentic workflows, workflows, workflow, create, file

Tasks maintaining the agentic workflow infrastructure itself: updating issue templates, failure reports, compiling workflow YAML, and wiring CI hooks. Highest success rate (81.6%) — highly specific and well-scoped tasks.

Representative PRs:

• ✅ #11053 — Update parent issue template for agentic-workflow failures
• ❌ #11054 — Auto-assign @copilot to workflow sync issues when agent token available
• ✅ #14041 — Add plugin installation support via frontmatter with test coverage

---

Cluster 4: Safe-Outputs & Project Infrastructure

102 tasks (11.3%) | Success: 76.5% | Avg commits: 4.7 | Avg files: 20.0

Top keywords: safe, project, safe outputs, outputs, safe output, output, create

Tasks configuring the safe-outputs MCP container (Dockerfile, git installation, node:lts images) and project-level infrastructure. Solid 76.5% success with the most commits per task (avg 4.7).

Representative PRs:

• ❌ #11116 — Fix create_pull_request in safe-outputs container by installing git
• ✅ #14062 — Fix dispatch_workflow to use PR branch ref and resolve merge conflicts
• ✅ #14171 — Fix smoke-claude: handle tool failures gracefully

---

Cluster 5: Workflow Failure Investigation

100 tasks (11.1%) | Success: 78.0% | Avg commits: 3.0 | Avg files: 8.1

Top keywords: workflow, failure, agent, section, report, failed, debug

Tasks investigating and fixing specific agent run failures — debugging daily-cli-performance, ANSI escape sequences in YAML, analyzing reports. 78% success, low file churn (avg 8.1 files).

Representative PRs:

• ❌ #11059 — Install Go toolchain in daily-cli-performance workflow
• ✅ #11060 — Merge maintenance jobs and add comprehensive logging
• ✅ #11068 — Prevent ANSI escape sequences in compiled workflow YAML files

---

Cluster 6: Investigation & Debugging

94 tasks (10.4%) | Success: 72.3% | Avg commits: 3.3 | Avg files: 26.2

Top keywords: reference, why, investigate, debug, review, see, comment

Investigation-heavy tasks where the agent is given context from prior issues/PRs and asked to debug or review. Wide file churn (avg 26.2 files). 72.3% success rate.

Representative PRs:

• ✅ #11082 — Fix markdown code region balancer treating indented examples as nested
• ✅ #11129 — Fix safe-outputs server startup by copying tools.json to expected location
• ✅ #14255 — Fix issue-monster workflow by enabling needs.* expression evaluation

---

Cluster 7: Campaign & Security Automation

87 tasks (9.6%) | Success: 56.3% | Avg commits: 4.3 | Avg files: 8.7

Top keywords: campaign, docs, security, alert, dependabot, project, prs

Tasks building and evolving the dependabot campaign system, security alert processing, and PR review automation. Second-lowest success rate (56.3%) — complex multi-component tasks with higher failure risk.

Representative PRs:

• ✅ #11070 — chore: campaign discovery via label-based approach
• ✅ #11080 — Clarify tracker-id is optional for campaign worker workflows
• ✅ #14225 — Refactor Dependabot Project Manager to process PRs by type

---

Cluster 8: CI Job Analysis & Go/TypeScript Fixes

49 tasks (5.4%) | Success: 75.5% | Avg commits: 3.4 | Avg files: 22.8

Top keywords: job, analyze workflow, job url, workflow, analyze, failing, logs

Smallest cluster (5.4%) — analyzing failing CI jobs by URL, fixing Go lint errors, and TypeScript type fixes. 75.5% success, highest additions per task (avg 1722) due to larger refactors.

Representative PRs:

• ✅ #11915 — Fix staticcheck S1009 lint error: remove redundant nil check on map
• ✅ #12206 — Fix TypeScript error in close_expired_discussions: add duplicateCount
• ✅ #13949 — Simplify permissions: agent job ALWAYS gets contents write

---

Sample Data Table (100 most recent PRs)

PR	Title	Cluster	Outcome	Commits	Files
#14275	(WIP) Improve test quality for threat detection	Code Quality & Testing	🔄 open	1	0
#14274	(WIP) Fix firewall SSL-bump configuration extraction	Code Quality & Testing	🔄 open	1	0
#14273	(WIP) Add documentation for health command in CLI	MCP & CLI Tooling	🔄 open	1	0
#14269	Revert gh-aw-mcpg to v0.0.103	MCP & CLI Tooling	✅ merged	3	149
#14268	Add workflow guidance and cross-references to CLI help	Code Quality & Testing	❌ closed	2	5
#14267	Verify @playwright/mcp version is already updated	MCP & CLI Tooling	❌ closed	1	0
#14266	Document SSL-bump feature for AWF firewall	Code Quality & Testing	✅ merged	2	1
#14265	(WIP) Revert gh-aw-mcpg version in constants.go	MCP & CLI Tooling	❌ closed	1	0
#14264	Revert MCP Gateway to v0.0.78	MCP & CLI Tooling	❌ closed	2	148
#14260	Add fuzzy matching suggestions for CLI	Code Quality & Testing	✅ merged	7	5
#14259	Investigation: CI failure #14239 is false alarm	CI Job Analysis & Go/TypeScript Fixes	❌ closed	1	0
#14258	Standardize trial command help text with Examples	Code Quality & Testing	❌ closed	2	1
#14257	Fix compiler obfuscation: Don't wrap static quoted	Code Quality & Testing	✅ merged	2	66
#14255	Fix issue-monster workflow by enabling needs.* expressions	Investigation & Debugging	✅ merged	3	113
#14253	Document Actions permission restrictions detected	MCP & CLI Tooling	✅ merged	3	1
#14227	Review Dependabot npm PRs for docs/package.json	Campaign & Security Automation	🔄 open	6	6
#14225	Refactor Dependabot Project Manager to process PRs by type	Campaign & Security Automation	✅ merged	4	2
#14222	Remove anonymous bash tool syntax, require explicit	Safe-Outputs & Project Infrastructure	✅ merged	8	33
#14221	Fix TestRuntimeSetupPreservesUserVersions false positive	CI Job Analysis & Go/TypeScript Fixes	✅ merged	4	4
#14220	Fix 403 error: Configure github-token for Dependabot	Investigation & Debugging	❌ closed	3	3
#14211	Add edit tool and full bash access to daily-cli-tools	Agentic Workflow Maintenance	✅ merged	2	2
#14209	Make upgrade command version check non-blocking	MCP & CLI Tooling	✅ merged	2	2
#14208	Wrap agent log rendering in collapsible details	MCP & CLI Tooling	✅ merged	2	4
#14201	Add network access to agentic-workflows MCP server	MCP & CLI Tooling	✅ merged	4	28
#14198	(WIP) Update agentic workflows server name	Agentic Workflow Maintenance	❌ closed	1	0
#14191	Format auto-added "Fixes #N" as bullet point in PR	Workflow Failure Investigation	✅ merged	2	1
#14189	Teach create workflow agent to discover CLI automation	Safe-Outputs & Project Infrastructure	✅ merged	6	2
#14184	Limit create-issue to 1 in daily-cli-tools-tester	Agentic Workflow Maintenance	✅ merged	2	2
#14183	(WIP) Fix MCP server permission denied error	MCP & CLI Tooling	❌ closed	3	30
#14182	Add Dependabot Project Manager workflow	Agentic Workflow Maintenance	✅ merged	9	2
#14173	Rename MCP server identifier from agentic_workflow	Investigation & Debugging	✅ merged	3	31
#14171	Fix smoke-claude: handle tool failures gracefully	Safe-Outputs & Project Infrastructure	✅ merged	5	3
#14168	Add daily exploratory testing workflow for CLI tools	Agentic Workflow Maintenance	✅ merged	4	3
#14167	(WIP) Implement daily exploratory testing for audit	Agentic Workflow Maintenance	❌ closed	1	0
#14156	Capture exit codes and stderr when gh CLI commands	Agentic Workflow Maintenance	✅ merged	5	3
#14150	Remove prompt file management functions from init	Agentic Workflow Maintenance	✅ merged	4	10
#14149	Add max-tokens and max-iterations execution bounds	MCP & CLI Tooling	❌ closed	3	10
#14147	Add daily concurrency analysis workflow for MCP	Agentic Workflow Maintenance	✅ merged	2	2
#14140	Add binary path detection for MCP server self-invocation	MCP & CLI Tooling	✅ merged	9	6
#14139	Refactor MCP server update tool to call Go function	MCP & CLI Tooling	❌ closed	3	9
#14129	Add unit tests for compiler_yaml_main_job.go	Code Quality & Testing	✅ merged	4	86
#14127	Fix daily-fact workflow action-tag to include missing	Workflow Failure Investigation	✅ merged	3	87
#14125	Fix add_comment to handle discussion numbers via fallback	Investigation & Debugging	✅ merged	2	2
#14121	Investigation: Daily Issues Report Generator failure	Workflow Failure Investigation	❌ closed	1	0
#14120	Fix gh-aw binary availability for user-defined steps	Workflow Failure Investigation	✅ merged	3	24
#14119	Fix textblob version check in Copilot PR NLP Analysis	Workflow Failure Investigation	✅ merged	2	2
#14114	Update CLI versions: Claude Code 2.1.34, Copilot 0.0.x	MCP & CLI Tooling	✅ merged	2	59
#14110	(WIP) Fix layout and logo text centering in Quickstart	Campaign & Security Automation	❌ closed	1	0
#14098	Consolidate monitoring pattern into ProjectOps	Campaign & Security Automation	❌ closed	2	6
#14095	Replace emojis with Starlight icons in documentation	Campaign & Security Automation	❌ closed	4	8
#14069	Remove payload-dir from frontmatter documentation	Campaign & Security Automation	✅ merged	2	1
#14062	Fix dispatch_workflow to use PR branch ref	Safe-Outputs & Project Infrastructure	✅ merged	3	2
#14057	(WIP) Debug workflow failure in agent performance	Workflow Failure Investigation	❌ closed	1	0
#14056	(WIP) Debug workflow failure in Workflow Normalizer	Workflow Failure Investigation	❌ closed	1	0
#14053	Add working directory logging and Docker workdir	MCP & CLI Tooling	✅ merged	5	30
#14044	Remove GH_TOKEN from agentic-workflows MCP server	Workflow Failure Investigation	✅ merged	5	146
#14041	Add plugin installation support via frontmatter	Agentic Workflow Maintenance	✅ merged	8	158
#14007	Add telemetry.enterprise.githubcopilot.com to copilot	MCP & CLI Tooling	✅ merged	3	100
#14003	Fix test assertions in parse_mcp_gateway_log.test	CI Job Analysis & Go/TypeScript Fixes	✅ merged	2	1
#13999	Remove duplicate filename display in displayFileContents	Investigation & Debugging	✅ merged	2	2
#13998	Integrate gh-aw-mcpg v0.0.103	MCP & CLI Tooling	✅ merged	3	146
#13997	Remove nested groups in displayDirectories	MCP & CLI Tooling	❌ closed	2	3
#13996	Add CLI build steps for dev mode in agentic-workflows	MCP & CLI Tooling	✅ merged	15	36
#13993	Integrate gh-aw-mcpg v0.0.113	MCP & CLI Tooling	❌ closed	2	59
#13984	Upgrade MCP Gateway to v0.0.101	Safe-Outputs & Project Infrastructure	❌ closed	2	146
#13980	fix: ensure /home/runner/.copilot directory has correct permissions	MCP & CLI Tooling	✅ merged	4	2
#13974	Add --cmd argument to agentic-workflows MCP server	Agentic Workflow Maintenance	✅ merged	4	32
#13969	Fix MCP parameter rendering for arrays and objects	Investigation & Debugging	✅ merged	4	3
#13966	(WIP) Fix failing GitHub Actions workflow test	CI Job Analysis & Go/TypeScript Fixes	✅ merged	2	1
#13956	Add serena container to predownload list	MCP & CLI Tooling	✅ merged	5	21
#13955	Add symbol search verification to smoke workflow	Agentic Workflow Maintenance	✅ merged	2	4
#13949	Simplify permissions: agent job ALWAYS gets contents write	CI Job Analysis & Go/TypeScript Fixes	✅ merged	7	5
#13948	Mount gh CLI binary in agentic-workflows MCP server	Agentic Workflow Maintenance	✅ merged	6	27
#13946	docs: add getting started videos to index landing	MCP & CLI Tooling	✅ merged	3	1
#13945	Add agent-generated files directory to artifact upload	Investigation & Debugging	✅ merged	5	146
#13942	fix: Remove duplicate Go module cache causing tar error	CI Job Analysis & Go/TypeScript Fixes	✅ merged	7	2
#13940	Remove italic styling from video captions	Workflow Failure Investigation	✅ merged	2	1
#13939	Add script to generate video poster images	Investigation & Debugging	✅ merged	5	5
#13936	(WIP) Downgrade Claude to version 2.1.29	MCP & CLI Tooling	❌ closed	1	0
#13935	Downgrade Claude Code CLI to 2.1.29	MCP & CLI Tooling	✅ merged	3	37
#13932	Remove tests asserting specific version constant values	CI Job Analysis & Go/TypeScript Fixes	✅ merged	5	90
#13931	Fix runtime-import file path resolution and checkout	Agentic Workflow Maintenance	✅ merged	6	149
#13925	Remove unnecessary test dependency from canary_go	CI Job Analysis & Go/TypeScript Fixes	✅ merged	2	1
#13920	Fix firewall SSL-bump field extraction in frontmatter	Code Quality & Testing	✅ merged	3	3
#13919	Bump CLI versions: Copilot 0.0.403, Codex 0.97.0	MCP & CLI Tooling	✅ merged	3	60
#13918	Add security rationale to permissions documentation	Code Quality & Testing	✅ merged	3	1
#13915	Add missing category field to workflows	Agentic Workflow Maintenance	❌ closed	2	6
#13914	Use workflow-id markers for close-older-discussions	Code Quality & Testing	✅ merged	4	4
#13907	Add issues:write permission to create-discussion job	Safe-Outputs & Project Infrastructure	✅ merged	3	50
#13906	(WIP) Fix fallback issue creation route for safe outputs	Safe-Outputs & Project Infrastructure	❌ closed	1	0
#13903	Configure PR triage reports with 1-day expiration	Workflow Failure Investigation	✅ merged	2	2
#13902	Add Video component for Astro Starlight documentation	Code Quality & Testing	✅ merged	5	3
#13901	Include .agents folder in sparse checkout	CI Job Analysis & Go/TypeScript Fixes	✅ merged	5	3
#13864	Add video embedding support via astro-embed	Campaign & Security Automation	❌ closed	5	4
#13860	Delete .github/aw/*.md files now downloaded from GitHub	Agentic Workflow Maintenance	❌ closed	6	17
#13859	Fix Serena container selection array handling bug	Workflow Failure Investigation	❌ closed	5	2
#13855	(WIP) Debug Copilot Agent prompt clustering analysis	Workflow Failure Investigation	❌ closed	4	11
#13854	Remove windows-386 from release build targets	MCP & CLI Tooling	✅ merged	2	2
#13853	Normalize file paths to Unix format in compiled .lock.yml	Agentic Workflow Maintenance	✅ merged	5	3

---

Key Findings & Recommendations

1. MCP/CLI tasks have the lowest success rate (57%) despite being the largest category (23%). Consider breaking large MCP upgrade tasks into smaller atomic steps and adding automated compile+test gates before merging.

2. Agentic Workflow Maintenance is the highest-success category (81.6%) — highly repetitive, well-defined tasks with explicit success criteria. Use this as a template for improving prompt quality in other categories.

3. Investigation tasks (Cluster 6) have high file churn (avg 26 files, 72.3% success) — providing more targeted context (specific file paths, error stacks, reproduction steps) in prompts would likely improve outcomes and reduce scope creep.

4. Campaign & Security Automation tasks have 56.3% success — multi-component changes are risky. Splitting into smaller focused PRs (schema changes, workflow changes separately) could improve merge rates.

5. Average 3–5 commits per task across all clusters — agents rarely solve problems in one shot. Adding structured acceptance criteria and expected outputs to prompts may reduce iteration counts.

References:

• Workflow run: §23765020979

AI generated by Copilot Agent Prompt Clustering Analysis · history

• expires on Mar 31, 2026, 8:39 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Agent Prompt Clustering Analysis.

A newer discussion is available at Discussion #23775.