[prompt-clustering] Copilot Agent Prompt Clustering Analysis - December 14, 2025 #6449
Closed
Replies: 1 comment
-
|
⚓ Avast! This discussion be marked as outdated by Copilot Agent Prompt Clustering Analysis. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
🔬 Copilot Agent Prompt Clustering Analysis
Analysis Date: 2025-12-14
Summary
This analysis applies NLP-based clustering to 879 task prompts from copilot agent PRs over the last 30 days, identifying 5 distinct clusters of task types. The overall success rate is 77.1% with an average task duration of 2.0 hours.
Key Findings: The most common task type is "Miscellaneous Tasks" (350 tasks), while "New Features & Implementation" tasks have the highest success rate at 80.0%. Tasks typically change 14.2 files with an average of 2 hours duration.
Full Analysis Report
General Insights
Most Common Task Type: Miscellaneous Tasks (350 tasks, 39.8% of total)
Highest Success Rate: New Features & Implementation (80.0%)
Most Time-Consuming: Bug Fixes & Error Resolution (avg 3.2 hours)
Cluster Analysis
Cluster 1: Miscellaneous Tasks
Top TF-IDF Terms: details original, details original issue, original issue resolve, issue resolve, section details
Characteristics:
This cluster performs above average with a 78.9% success rate.
Tasks in this cluster complete faster than average.
These tasks typically involve fewer file changes.
Example PRs:
Sample Prompts
Cluster 2: New Features & Implementation
Top TF-IDF Terms: update, github, file, run, code
Common Keywords: update, add, test, create, ui
Characteristics:
This cluster performs below average with a 74.0% success rate.
Tasks in this cluster complete faster than average.
These tasks typically involve more extensive file changes.
Example PRs:
Sample Prompts
Cluster 3: New Features & Implementation
Top TF-IDF Terms: workflow, agentic, agentic workflow, daily, create
Common Keywords: create, update, add, ui, test
Characteristics:
This cluster performs above average with a 80.0% success rate.
Tasks in this cluster complete faster than average.
These tasks typically involve fewer file changes.
Example PRs:
Sample Prompts
Cluster 4: Bug Fixes & Error Resolution
Top TF-IDF Terms: add, command, firewall, compile, field
Common Keywords: add, update, test, ui, error
Characteristics:
This cluster performs below average with a 76.5% success rate.
Tasks in this cluster take longer than average.
These tasks typically involve fewer file changes.
Example PRs:
Sample Prompts
Cluster 5: Bug Fixes & Error Resolution
Top TF-IDF Terms: fix, tests, docs, javascript, files
Common Keywords: fix, test, lint, ui, error
Characteristics:
This cluster performs above average with a 80.0% success rate.
Tasks in this cluster complete faster than average.
These tasks typically involve fewer file changes.
Example PRs:
Sample Prompts
Success Rate by Cluster
Key Findings
1. Task Distribution: The largest cluster is "Miscellaneous Tasks" with 350 tasks (39.8% of all tasks). This suggests that copilot agents are most frequently used for miscellaneous tasks.
2. Success Patterns: 3 out of 5 clusters have success rates above 78%. Tasks in the "New Features & Implementation" cluster show the highest success rate at 80.0%, indicating these types of tasks are well-suited for agent automation.
3. Complexity Analysis: The "Bug Fixes & Error Resolution" cluster takes the longest to complete (avg 3.2 hours), suggesting higher complexity. However, its success rate of 76.5% shows that even complex tasks can be successfully automated.
Recommendations
1. Optimize for High-Success Task Types: Tasks in the "New Features & Implementation" category show 80.0% success rate. Consider templating or documenting best practices for these types of prompts to maintain high quality.
2. Improve "New Features & Implementation" Tasks: With a 74.0% success rate, tasks in this cluster may benefit from more detailed prompts, better context, or task decomposition.
3. Optimize Time-Intensive Tasks: "Bug Fixes & Error Resolution" tasks take 3.2 hours on average. Consider breaking these into smaller subtasks or providing more specific guidance to reduce iteration time.
4. Prompt Engineering Insights: Analysis of TF-IDF terms reveals that successful prompts often include specific technical terms and clear action verbs. Encourage prompt writers to be explicit about the desired outcome and provide relevant context.
Visualizations
The following visualizations are available as workflow artifacts:
Analysis generated by NLP Prompt Clustering workflow on 2025-12-14 19:20 UTC
Beta Was this translation helpful? Give feedback.
All reactions