[copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-03-26

[github-actions[bot]]

Executive Summary

• Sessions Analyzed: 50
• Analysis Period: 2026-03-26 (07:23–10:09 UTC)
• Active Copilot Branches: 3
• Copilot Agent Runs: 2 (100% success rate today)
• Overall Session Completion Rate: 16.0%
• Experimental Strategy: None (standard analysis run)

---

Key Metrics

Metric	Value	Trend
Total Sessions	50	→
Successful	8 (16%)	→
Failed	2 (4%)	↑
Skipped	12 (24%)	→
Action Required	24 (48%)	→
Cancelled	4 (8%)	→
Copilot Agent Runs	2	↓
Copilot Agent Success	2 (100%)	↑
Copilot Avg Duration	8.85 min	→
All-time Copilot Success Rate	64.8% (59/91)	↓
Last 7-day Copilot Success Rate	56.0% (14/25)	↓

---

📈 Session Trends Analysis

Completion Patterns

Completion rates have shown a sharp decline after the strong period of late February/early March 2026, where rates routinely hit 80–100%. Since mid-March, daily rates have dropped to single digits on most days, with the all-time 14-day average now at 20.1% vs. the all-time average of 43.3%. The predominance of action_required conclusions (review agents awaiting human merge decisions) is the primary structural driver of the low overall rate — this is expected behavior, not a failure signal.

Duration & Efficiency

Copilot agent session durations remain relatively stable around a median of 9.6 min. The notable outlier on 2026-02-27 (40.3 min) appears to be a one-off long-running task. Today's two sessions (11.8 min and 5.9 min) are well within the normal range. The all-time mean duration is 11.0 min, suggesting consistent task scope.

---

Branch Activity — Today

copilot/refactor-write-count-calculation — 27 sessions, 79.8 min window

• Copilot Agent: Running Copilot coding agent — ✅ success (11.8 min, 08:49–09:01 UTC)
• CI: 2× cancelled (superseded by subsequent pushes)
• Content Moderation: ✅ success
• AI Moderator: ❌ failure — notable: moderation agent failed on this branch
• Doc Build - Deploy: ✅ success
• Grumpy Code Reviewer: ✅ success (round 1)
• Full review chain (Security, Grumpy, /cloclo, Scout, Q, PR Nitpick): 2× action_required (rounds 2 & 3)
• Pattern: Double review round on refactor branch — agent's first push triggered review, a revision triggered a second full chain. Awaiting human merge decision.

copilot/sub-pr-23074 — 12 sessions, 28.4 min window

• Copilot Agent: Addressing comment on PR #23075 — ✅ success (5.9 min, 08:11–08:17 UTC)
• CI: 2× cancelled
• Doc Build - Deploy: ✅ success
• Grumpy Code Reviewer: ✅ success
• Full review chain (Scout, Security, /cloclo, Q, PR Nitpick, Grumpy): 1× action_required
• Pattern: PR comment response — short task, single agent run, standard review chain fired.

copilot/fix-env-expression-rejection — 8 sessions, 3.1 min window

• Copilot Agent: None visible in analyzed window
• CI: ❌ failure
• Grumpy Code Reviewer: ✅ success
• Full review chain (Scout, Grumpy, Q, /cloclo, PR Nitpick, Security): action_required
• Pattern: CI failure followed by full review chain firing — copilot agent may have run outside the analysis window. Branch status: pending CI fix.

---

Success Factors ✅

1. PR Comment Response Pattern: Short, focused tasks (PR comment responses) continue to have very high success rates. Today's 5.9 min response to PR fix: remove double-counted write_actions from partially_reducible evidence; rename Minutes → Action Minutes #23075 is a clean example — single-shot completion with no retries.

   • Success rate for this pattern: historically ~85–90%

2. Refactor Tasks with Clear Scope: The refactor-write-count-calculation task completed in 11.8 min with no agent retries. Refactor branches with specific, bounded change targets ("write count calculation") tend to succeed in single runs.

3. Consistent Duration Profile: Agent runs today (5.9 and 11.8 min) align closely with the historical median (9.6 min), suggesting the agent is not experiencing excessive loops or context confusion.

---

Failure Signals ⚠️

1. AI Moderator Failure on Refactor Branch: The AI Moderator workflow failed on copilot/refactor-write-count-calculation. This is an unusual signal — suggests the PR content may have triggered moderation heuristics. Worth monitoring on future refactor branches to determine if this is a false positive or a structural issue with the PR description format.

   • Observed frequency: occasional (noted on ~10% of past analysis days)

2. CI Failures on fix-env-expression-rejection: CI failed without a visible copilot agent response in the analysis window. This may indicate the agent has not yet been triggered to address the failure, or that the fix requires investigation beyond automated correction.

3. Declining 7-day Copilot Success Rate: The last 7-day rate (56.0%) is notably below the all-time rate (64.8%). Recent days (2026-03-23 to 2026-03-25) showed in_progress sessions at analysis time and low success counts. Some of this is measurement artifact (sessions captured mid-run), but the trend warrants continued monitoring.

---

Notable Observations

Loop Detection

• Sessions with loops today: 0 identified
• The double review round on refactor-write-count-calculation is expected behavior (push → review → revision push → review again), not a loop

Tool Usage Patterns (inferred from session metadata)

• Most active workflow agents: Grumpy Code Reviewer, Scout, /cloclo, Q, PR Nitpick Reviewer, Security Review Agent
• All review agents fired in parallel (same created_at timestamp) — efficient parallelism
• CI cancellations on superseded commits are working as expected

Moderation Activity

• AI Moderator failure + Content Moderation success on same branch: suggests the content moderation pipeline has redundancy. The Content Moderation agent succeeded while AI Moderator failed — likely a transient failure or backend issue.

---

Trends Over Time

Period	Copilot Success Rate	Avg Completion Rate
All-time (32 days)	64.8% (59/91)	43.3%
Last 14 days	~48%	20.1%
Last 7 days	56.0% (14/25)	8.3%
Today	100% (2/2)	16.0%

The last 7-day copilot success rate (56%) is 8.8 points below the all-time average. However, today's 100% success breaks a recent streak of low-activity/in-progress days. The very low overall completion rates (8–16%) in the last 7 days are primarily driven by the high volume of action_required review agent sessions awaiting human action — not actual failures.

---

Actionable Recommendations

For Users Writing Task Descriptions

1. Use specific, bounded scope descriptions: Branch names like refactor-write-count-calculation correlate with single-shot success. Vague scope (e.g., fix-* without context) risks CI failures without clear recovery paths.
2. Include PR links in comment-response tasks: The Addressing comment on PR #23075 pattern provides the agent with unambiguous context, enabling fast (5.9 min) completions.

For System Improvements

1. AI Moderator reliability: Investigate the failure on refactor-write-count-calculation. If this is a false positive rate issue, consider raising the threshold or adding a retry mechanism.
2. CI failure triage visibility: When CI fails on a copilot branch without a subsequent agent run in the window, a signal or notification would help identify whether the agent will retry or if human intervention is needed.

---

Statistical Summary

Total Sessions Analyzed:       50
Successful:                     8 (16.0%)
Failed:                         2  (4.0%)
Skipped:                       12 (24.0%)
Action Required:               24 (48.0%)
Cancelled:                      4  (8.0%)

Copilot Agent Sessions:         2
Copilot Agent Success:          2 (100%)
Copilot Agent Avg Duration:  8.85 min

All-time Copilot Success:   64.8% (59/91 over 32 days)
Last 7-day Copilot Success: 56.0% (14/25)

Historical Avg Completion Rate: 43.3%
Recent 14-day Avg:              20.1%

---

Next Steps

• Monitor AI Moderator failure pattern on future refactor branches
• Track fix-env-expression-rejection — does a copilot agent run address the CI failure?
• Investigate whether the 7-day copilot success rate decline is measurement artifact (in_progress captures) or real
• Consider alerting when no copilot agent run fires within N minutes of a CI failure on a copilot branch

---

References:

• §23592142699 — This analysis workflow run
• §23588737676 — refactor-write-count-calculation sessions

AI generated by Copilot Session Insights · history

• expires on Mar 27, 2026, 11:49 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Session Insights.

A newer discussion is available at Discussion #23230.