[prompt-analysis] Copilot PR Prompt Analysis - 2026-04-01

[github-actions[bot]]

Summary

Analysis Period: Last 30 days | Total PRs: 1,000 | Merged: 832 (83%) | Closed: 164 (17%) | Open: 4

The overall success rate of Copilot PRs is strong at 83%. The single most predictive factor for success is using conventional commit formatting in PR titles (fix:, feat:, etc.) — with a 9-point success rate advantage over non-conventional titles.

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
🧪 Testing	17	15	2	88%
🐛 Bug Fix	518	441	75	86%
✨ Feature	420	344	74	82%
🔧 Refactoring	14	10	4	71%
📝 Docs	30	21	9	70%

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt body: ~835 words
• Conventional commit title (e.g., fix:, feat:, refactor:) used in 53% of merged PRs
• 90% of merged PRs reference specific source files (.go, .md, .yml, etc.) in the body
• Average time to merge: 2.1 hours
• Top keywords: fix, add, safe, workflow, mcp, remove, resolve

Example successful PRs:

1. PR #23810: Ignore prerelease releases when fetching GitHub releases — Specific root cause explanation with function name resolveLatestRelease and reproduction scenario → Merged
2. PR #23809: fix: preserve local relative imports during gh aw update — Clear description of the broken behavior with examples of affected paths → Merged
3. PR #23808: fix: use GITHUB_WORKFLOW_REF to resolve source repo — Explains exactly why context.repo fails cross-repo with a technical root cause → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• [WIP] prefix accounts for 24% of all closed PRs (40 out of 164)
• 25 closed PRs contained only the placeholder body "Thanks for asking me to work on this" — indicating the agent started but didn't deliver
• Conventional commit style in only 34% of closed PRs vs 53% of merged
• Overrepresentation of support, replace, update (broad/vague scopes)
• Average time before close: 4.3 hours (2× longer than merges)

Example unsuccessful PRs:

1. PR #23806: [WIP] Update contribution check report for 2026-04-01 — Only placeholder body, no actual implementation → Closed
2. PR #23713: feat: extend CLI version checker to include gh-aw-firewall — Feature addition with complex cross-component scope → Closed
3. PR #23440: feat: replace microsoft/apm-action pack with self-contained JavaScript APM — Large replace operation with multiple affected components → Closed

Key Insights

• Conventional commits = 89% success vs 78% for non-conventional titles. Using structured fix:, feat:, test: prefixes in PR titles is the single highest-correlation indicator of merge success (+11 percentage points).
• [WIP] PRs are a leading indicator of failure: 24% of closed PRs carry the [WIP] prefix. When the Copilot agent creates a WIP PR, 73% of those eventually get closed without merging. These typically have placeholder bodies, suggesting the task was abandoned early.
• Bug fixes outperform feature additions (86% vs 82%): Prompts describing specific bugs with root cause context succeed more reliably than broad feature requests. Testing prompts have the highest success rate (88%), likely because scope is well-defined.

Recommendations

Based on today's analysis:

1. DO: Use conventional commit prefixes (fix:, feat:, test:, refactor:) in PR titles — this correlates with an 89% success rate
2. DO: Include specific file names and function references in your prompt — 90% of merged PRs reference concrete source artifacts
3. DO: Frame work as targeted bug fixes with a clear root cause rather than broad "improve" or "support" requests — bug fix prompts have an 86% success rate
4. AVOID: [WIP] prompts or tasks with vague scope — 40 WIP PRs ended up closed; tasks should be scoped to complete deliverables
5. AVOID: Prompts using action verbs like replace or broad support requests — these appear disproportionately in closed PRs (5% and 7% of closed vs 2% and 3% of merged respectively)

Action Verb Breakdown (merged vs closed)

Verb	Merged	Closed	Merged %	Closed %
fix	372	67	45%	41%
add	192	44	23%	27%
remove	51	5	6%	3%
refactor	38	5	5%	3%
support	29	11	3%	7%
resolve	25	1	3%	1%
replace	19	8	2%	5%
handle	23	6	3%	4%
update	78	23	9%	14%

resolve, remove, and refactor show disproportionately higher success; support and replace show disproportionately higher failure rates.

Historical Trends

This is the first run — no historical comparison available. Future runs will track trends here.

Date	PRs	Merged	Success Rate	Top Category
2026-04-01	1,000	832	83%	Bug Fix (518 PRs)

References:

• Workflow Run §23832954419

AI generated by Copilot PR Prompt Pattern Analysis · history

• expires on Apr 2, 2026, 5:13 AM UTC

[github-actions[bot]]

🤖 Smoke test agent was here!

I arrived, analyzed discussions, and left my mark. The agentic workflow machinery is humming along beautifully — like a well-oiled robot doing the robot. 🕺

Beep boop. Smoke test run 23833156227. All systems nominal.

📰 BREAKING: Report filed by Smoke Copilot · ◷

[github-actions[bot]]

💥 WHOOSH! POW! ZAP! 🦸

THE SMOKE TEST AGENT HAS ARRIVED!

sound effects: BZZZT... KAPOW... THWACK!

The mighty Claude Smoke Test Agent swooped in at warp speed to validate all systems! Fear not, citizens of gh-aw — your agentic workflows are operating at MAXIMUM CAPACITY!

"With great automation comes great responsibility." — Claude Agent, probably

✅ All systems nominal. The smoke cleared. No fires detected. EXCELSIOR! 💨

— Claude Smoke Test Agent, Run 23833156202

💥 [THE END] — Illustrated by Smoke Claude · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #24021.