Agent Persona Exploration - 2026-03-31

[github-actions[bot]]

Persona Overview

• Agent tested: developer.instructions (agentic-workflows custom agent)
• Scenarios tested: 7 (across 5 software personas)
• Average Quality Score: 4.94/5.0
• Run: §23779329400

Key Findings

• Consistently high quality: All 7 scenarios scored 4.8–5.0; the agent produces production-ready configurations without prompting for clarification
• Security-first by default: strict: true, minimal permissions, and safe-outputs write separation appeared in every response
• Engine selection is well-calibrated: Claude chosen for reasoning-heavy tasks (log analysis, visual comparison, rubric-based triage); Copilot for GitHub-API-heavy and structured output tasks
• Prompt engineering is exemplary: Multi-step structured prompts with concrete commands, thresholds, and output templates — not vague instructions
• Minor permission inconsistency: 2 of 7 PR-automation scenarios specified pull-requests: write directly on the agent job, while the more secure pattern (read-only agent job, write delegated entirely to safe-outputs) appeared in the other 5

Top Patterns

1. Most common trigger: pull_request with paths: filter + concurrency.cancel-in-progress: true (5/7 scenarios)
2. Most recommended tools: github MCP toolsets always present; bash with explicit command allowlist (6/7); engine-specific: playwright MCP (visual regression), agentic-workflows MCP audit tool (deployment failures)
3. Security practices observed: strict: true in 100% of responses; network allowlist always scoped to minimum needed; safe-outputs used as the sole write path; noop declared as mandatory exit in every scenario

View High Quality Responses (Top 3)

do-1 — DevOps Deployment Failure Analyzer (5.0/5)
Standout: Used the agentic-workflows MCP audit tool itself for structured log analysis, showing meta-awareness of the ecosystem. Included dedup check for existing incidents, close-older-issues: false for audit history, expires: 30d for cleanup, and a production hardening table with Slack, oncall assignment, and repo-memory enhancements.

qa-2 — QA Bug Triage (5.0/5)
Standout: Only scenario to use add-labels safe-output with an explicit allowed: list — preventing the agent from polluting the label namespace. Also included rate-limit: max: 3 / window: 60 to prevent API flooding on bulk issue imports. Guard condition for already-triaged issues (existing severity/* label → noop).

be-1 — Backend DB Schema Review (5.0/5)
Standout: Most explicit about the permission model — pull-requests: read on the agent job with a comment explicitly noting the framework injects write access for safe-outputs separately. Step-by-step SQL severity classification table (CRITICAL/HIGH/MEDIUM/INFO) with specific patterns like ADD COLUMN ... NOT NULL without DEFAULT.

View Areas for Improvement

Inconsistent permission model for PR-automation workflows

• fe-1 (visual regression) and qa-1 (coverage analysis) specified pull-requests: write on the agent job
• be-1, be-2, qa-2, do-1, pm-1 correctly used read-only agent permissions, delegating writes entirely to safe-outputs
• Not a blocking issue (the framework may accept either form), but the safer pattern should be standardized

be-2 adds Go module proxy to network allowlist

• To install oasdiff via go install, the response added proxy.golang.org and sum.golang.org to the network allowlist
• This is technically correct but introduces an external dependency on every run; the response does suggest pre-installing in a custom runner image as mitigation

Recommendations

1. Clarify the PR-automation permission model in .github/aw/create-agentic-workflow.md: document whether pull-requests: write belongs on the agent job or should be entirely delegated to safe-outputs. The inconsistency across 7 responses suggests the documentation is ambiguous on this point.

2. Promote the add-labels allowlist pattern in workflow examples — it's the only safe way to let an agent apply labels without risking namespace pollution, but it only appeared in 1/7 scenarios (the bug triage case where it was obviously needed).

3. Add a rate-limit usage example to the documentation for issue-triggered workflows — the rate-limit: max: 3 / window: 60 pattern from qa-2 elegantly prevents API flooding on bulk operations but is easy to miss if not shown in an example.

References:

• §23779329400

AI generated by Agent Persona Explorer · history

[mgttt]

[deleted by mistake — please ignore]

[github-actions[bot]]

This discussion has been marked as outdated by Agent Persona Explorer.

A newer discussion is available at Discussion #23815.