[prompt-analysis] 🤖 Copilot PR Prompt Pattern Analysis - 2025-11-24

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2025-11-24

This analysis examines 1000 Copilot-generated PRs from the last 30 days to identify patterns that correlate with successful merges versus closed PRs.

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 773 (77.3%) | Closed: 225 (22.5%) | Open: 2

Full Analysis Details

Prompt Categories and Success Rates

| Bug Fix | 988 | 766 | 77.5% |
| Feature | 949 | 738 | 77.8% |
| Documentation | 896 | 697 | 77.8% |
| Testing | 829 | 649 | 78.3% |
| Refactor | 432 | 334 | 77.3% |
| Other | 1 | 0 | 0.0% |

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 469 words
• Most common keywords: copilot (5278), agent (4247), workflow (4001), github (3734), coding (3380)
• Success rate across categories: 77-78%

Example successful prompts:

• PR Fix null author handling in close-old-discussions workflow #4638: The jq filter in the discussion fetch step failed when discussions had null authors, preventing the workflow from matchi...
• PR Fix: Create temp directory before custom steps #4637: Custom steps that reference /tmp/gh-aw/ fail because the directory is created after custom steps execute. ## Changes ...
• PR Optimize close-old-discussions workflow with pre-filtered data #4636: The close-old-discussions workflow made 2+ paginated GitHub API calls during agent execution, transferring ~100 discussi...

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 466 words
• Most common keywords: copilot (1564), workflow (1228), agent (1212), github (1108), coding (989)
• Often involve assumptions or investigations that don't lead to actionable changes

Example unsuccessful prompts:

• PR [WIP] Add automatic init command when using add #4600: Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a pla...
• PR Replace gh agent-task extension with agentic-workflows MCP tool #4562: The copilot-session-insights workflow depends on the gh agent-task CLI extension, which is Enterprise-only and requi...
• PR Verify close-discussion safe output implementation is complete #4548: The requested "close-discussion" safe output type already exists in the codebase with full implementation across all lay...

Key Insights

1. Consistent Success Pattern: All prompt categories show similar success rates (~77-78%), suggesting that Copilot PRs have a baseline merge rate around 77.5% regardless of task type.
2. Prompt Length Neutral: Merged and closed PRs have nearly identical average prompt lengths (469 vs 466 words), indicating length is not a determining factor for success.
3. Testing Performs Best: Testing prompts have the highest success rate at 78.3% (649/829 merged).

Recommendations

1. Focus on Testing PRs: Testing-related prompts show the highest success rate. When appropriate, frame prompts around test improvements or additions.
2. Maintain Clear Task Descriptions: Since success rates are consistent across categories, clarity and specificity in the task description appear more important than the category itself.
3. Standard Prompt Length Works: Both successful and unsuccessful PRs average ~465-470 words. Aim for comprehensive but focused prompts without over-explaining.

Historical Trends

Date	PRs	Success Rate	Top Category
2025-11-18	1000	77.0%	Update
2025-11-18	1000	77.4%	Bug Fix
2025-11-19	892	76.9%	Other
2025-11-20	1000	76.7%	Feature
2025-11-21	1000	76.9%	Bug Fix
2025-11-22	1000	77.4%	Feature
2025-11-23	1000	77.4%	Feature

Trend: Analyzing 7 days of data shows consistent PR success rates.

---

Generated by Copilot PR Prompt Analysis

AI generated by Copilot PR Prompt Pattern Analysis

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 1 week ago.