Agent Persona Exploration - 2026-04-02

[github-actions[bot]]

This report summarizes a systematic exploration of the agentic-workflows custom agent across 7 representative software worker personas and automation scenarios, conducted on 2026-04-02.

Persona Overview

• Agent tested: developer.instructions (agentic-workflows / create mode)
• Scenarios tested: 7 (across 5 personas)
• Average quality score: 4.8 / 5.0
• Workflow run: §23882503037

Key Findings

• 🏆 Prompt clarity and completeness scored perfectly (5.0/5.0) — every response included structured phase-based prompts, security tables, full frontmatter, and actionable examples
• 🔐 Security posture is consistently strong — strict: true, network: defaults, minimal permissions, and safe-outputs write isolation were applied in all 7 scenarios
• ⚠️ Tool selection is the weakest dimension (4.43/5.0) — some responses used broad bash tools; explicit tool enumeration (as in be-1) is the superior pattern
• 🔄 Guard clause / early-exit pattern was used well in issue triggers (qa-2, be-1) but could be more prominently documented
• 🚨 One possible hallucination detected: the roles: field in the release notes response — not a documented gh-aw frontmatter field

Top Patterns Observed

1. Trigger selection was highly accurate: workflow_run for deployment events, issues for bug triage, pull_request with paths: for component-scoped PRs, schedule for digests
2. Safe-outputs used correctly as the write layer: all write operations (comments, labels, issues, discussions) routed via safe-outputs — never via direct issues: write permission on the agent job
3. Anti-hallucination instructions emerged in the weekly digest response: "Never fabricate PR content — only use data returned by the API" — a notable prompt quality indicator
4. concurrency: cancel-in-progress: true appeared in be-1 as a standout pattern for PR-triggered workflows to prevent comment duplication on force-pushes
5. Prerequisite documentation appeared in fe-1 and qa-1 — proactively telling users what repo configuration is required before the workflow can succeed

View High Quality Responses (5.0/5.0)

do-1 — DevOps: Deployment Failure Analyzer (5.0)

• Perfect workflow_run trigger scoped to named deployment workflows
• workflow_dispatch secondary trigger for manual re-analysis
• Comprehensive RCA prompt: gather evidence → analyze logs → classify failure type → create structured issue
• max: 5 cap on issue creation prevents spam; close-older-issues: false preserves failure history
• Network defaults only — logs stay within GitHub's infrastructure

qa-2 — QA: Bug Triage Agent (5.0)

• issues: opened, labeled trigger with guard clause prevents acting on non-bug issues
• Pure GitHub toolsets — no bash needed, reducing attack surface
• Label allowlist: [p0, p1, p2, p3] is the critical security control — agent cannot invent labels
• Re-triage loop prevention explicitly analyzed (labeled trigger won't loop after severity label applied)
• Complete noop fallback for already-triaged issues

View Areas for Improvement

Tool selection breadth (4.43/5.0 avg)

• fe-1, pm-1, and qa-1 used broad bash tool lists where more specific enumeration would reduce the attack surface
• Best practice demonstrated in be-1: explicitly list ["grep", "awk", "sed", "cat", "diff"] rather than bash: true
• Recommendation: add an example of precise tool enumeration to .github/aw/github-agentic-workflows.md

Possible hallucinated frontmatter field

• pm-2 (release notes) included roles: [admin, maintainer] under on: push: tags: — this field does not appear in the gh-aw schema documentation
• The intent (restricting tag-triggered runs to trusted actors) is valid, but the implementation may be incorrect
• Recommendation: document approved patterns for restricting tag triggers in .github/aw/create-agentic-workflow.md

Paths filter inconsistency

• be-1 stated that paths: filter is not supported in frontmatter triggers and handled filtering inside the agent prompt instead
• This appears to be incorrect — paths: is a standard GitHub Actions trigger feature
• Recommendation: add a clear example of on: pull_request: paths: to the workflow creation guide

Recommendations

1. Document the guard clause / early-exit pattern in .github/aw/create-agentic-workflow.md — every issue/label-triggered workflow should show the noop early-exit as a first-class pattern with an example
2. Add a "bash tool enumeration" best practice in .github/aw/github-agentic-workflows.md — show explicit ["grep", "awk"] enumeration as preferred over bash: true, with security rationale
3. Add a scheduled workflow guide in .github/aw/ (e.g., scheduled-digest.md) covering the digest/release-notes pattern: cron trigger + workflow_dispatch, discussions: write permission, anti-hallucination instructions, and max: 1 discussion cap

References:

• §23882503037

AI generated by Agent Persona Explorer · history