[prompt-analysis] Copilot PR Prompt Analysis - 2026-03-22

[github-actions[bot]]

Summary

Analysis Period: Last 30 days (rolling window, up to 1,000 PRs from copilot/* branches)
Total PRs: 1,000 | Merged: 825 (83%) | Closed: 174 (17%) | Open: 1 (<1%)

The overall merge rate of 83% continues an upward trend from 79% a week ago — indicating improving prompt quality and workflow reliability in the github/gh-aw repository.

---

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
🔧 Bug Fix	770	642	128	83%
📝 Documentation	85	70	15	82%
🧪 Testing	100	78	21	79%
✨ Feature Addition	27	20	7	74%
🔨 Refactoring	9	8	1	89%
📦 Other	9	7	2	78%

Note: Refactoring shows the highest success rate (89%) though sample size is small (9 PRs). Bug fixes dominate the dataset at 77% of all PRs, suggesting this repo heavily focuses on incremental improvements.

---

✅ Successful Prompt Patterns

Common characteristics in merged PRs (825 total):

• Average prompt body: ~838 words
• 95% reference specific file paths (e.g., pkg/workflow/compiler.go, actions/setup/js/*.cjs)
• 75% include structured markdown sections (## Changes, ## What, ## How, ## Description)
• Prompts diagnose the root cause before proposing a fix

Example Successful Prompts

PR #22227 — Add host.docker.internal networking guidance to playwright prompt → Merged

"The playwright prompt injected by the compiler had no guidance on Docker networking, leaving agents to guess the correct hostname when navigating to local services. Since the Playwright MCP runs inside Docker..."

PR #22200 — Debug: Daily Community Attribution Updater failure — jq null accumulator in closing refs index → Merged

"The Daily Community Attribution Updater workflow failed because a jq reduce expression set the accumulator to null when any PR had an empty closingIssuesReferences array, causing the subsequent..."

PR #22197 — Use compact GitHub references (#N) in community contributions section → Merged

"The community contributions section in README.md used verbose full URLs with explicit issue titles as link text. GitHub auto-expands #N references natively, making the full URLs redundant noise..."

Pattern observed: Each successful prompt starts with a clear description of the problem, identifies the specific location in code, and explains the impact before describing the fix.

---

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs (174 total):

• Average prompt body: ~889 words (slightly longer than merged — more words ≠ better)
• WIP-labeled PRs: 52 total — only 33% merged vs 85% for non-WIP (most significant differentiator!)
• Closed PRs more commonly address competing/overlapping concerns already resolved elsewhere

Example Unsuccessful Prompts (Closed PRs)

PR #22205 — Fix CRITICAL toolName shell injection in mcp_setup_generator.go → Closed

"toolName from mcp-scripts: frontmatter was interpolated directly into shell cat > heredoc commands in compiled GitHub Actions YAML, enabling supply-chain injection..."
(Well-written prompt; likely closed because a better fix was developed in a concurrent PR)

PR #22194 — Remove experimental tag on mcp-scripts → Closed

"mcp-scripts is no longer experimental and should not emit a compiler warning..."
(Superseded by another approach)

PR #22171 — fix(issue-monster): lower min-integrity to none so cookie-labeled issues are readable → Closed

(Closed likely due to design decision reversal or merged via different approach)

---

Key Insights

• 🚩 WIP prefix is a strong failure signal: PRs with WIP in the title have only a 33% merge rate vs 85% for non-WIP. Copilot agents should only open PRs when work is complete, never as drafts or work-in-progress markers.

• 📁 Structured body sections improve outcomes: PRs with ## Changes, ## Description, or similar structured markdown sections have an 85% merge rate vs 76% for unstructured bodies — a 9-point improvement. Prompt templates with clear sections produce better results.

• 🎯 File-specificity matters, but modestly: Prompts referencing specific files/packages achieve 83% merge rate vs 78% for vague prompts — every prompt should name the files to be changed.

---

Recommendations

1. DO: Structure prompts with clear sections (## Problem, ## Changes, ## Why) — boosts merge rate by ~9 points
2. DO: Name the specific files, functions, or packages to be modified — this signals high-quality problem understanding
3. DO: Describe the root cause before the fix — narrows the solution space and reduces back-and-forth
4. AVOID: Using WIP in PR titles — these are closed at 67% rate. Open PRs only when ready for review.
5. AVOID: Long, padded prompts — closed PRs are actually slightly longer on average; quality and specificity matter more than length

---

Historical Trends (Last 7 Days)

Date	PRs	Merged	Closed	Success Rate
2026-03-14	1,000	790	209	79%
2026-03-15	1,000	797	201	79%
2026-03-16	1,000	803	196	80%
2026-03-17	1,000	812	185	81%
2026-03-18	1,000	818	179	81%
2026-03-19	1,000	816	180	81%
2026-03-21	1,000	821	177	82%
2026-03-22	1,000	825	174	83% ✅

Trend: Steady improvement over the past week, with merge rate rising from 79% → 83% (+4 points). At this pace, the repo is on track to exceed 85% merge rate within 2 weeks if current practices continue.

---

References:

• §23396116782

AI generated by Copilot PR Prompt Pattern Analysis · history

• expires on Mar 23, 2026, 5:10 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #22379.