[Pelis Agent Factory Advisor] Pelis Agent Factory Advisor — Workflow Analysis & Recommendations (2026-03-29)

[github-actions[bot]]

📊 Executive Summary

The gh-aw-firewall repository demonstrates exceptional agentic workflow maturity — one of the most complete agentic workflow deployments observed in a production repository. With 22 compiled agentic workflows spanning security, testing, documentation, CI/CD, and issue management, it already embodies most Pelis Agent Factory best practices. The primary gaps are a missing issue triage/labeling agent, a meta-audit/observability agent, and a domain-specific allowlist advisor tailored to this tool's unique security focus.

---

🎓 Patterns Learned from Pelis Agent Factory

From the Documentation Site

Key patterns from Peli's Agent Factory (100+ workflows in production):

Pattern	Description	Present Here?
Issue Triage	Auto-label and comment on new issues	❌ Missing
Continuous Code Quality	Refactoring, simplicity, style agents on schedule	❌ Missing
CI Doctor	Diagnoses CI failures, proposes fixes (69% merge rate in factory!)	✅ ci-doctor
Meta-Audit Agent	Monitors other agents' performance & costs	❌ Missing
Metrics Collector	Tracks daily performance across agent ecosystem	❌ Missing
Breaking Change Checker	Catches backward-incompatible changes pre-release	❌ Missing
Schema Consistency Checker	Detects drift between code, schemas, and docs	~Partial (cli-flag-consistency-checker)
Mergefest	Auto-merges main into PR branches	❌ Missing
Interactive ChatOps (Q)	On-demand slash command workflows (78% merge rate)	~Partial (/plan only)
Issue Arborist	Links related issues as sub-issues	❌ Missing
Secrets Scanning	Daily credential exposure scanning	✅ secret-digger-* (3 engines!)
Security Compliance	Vulnerability campaigns with SLAs	✅ dependency-security-monitor
Smoke Tests	Validates agent functionality on schedule	✅ smoke-* (4 workflows)

From the githubnext/agentics Repository

The agentics repo features: daily-repo-goals, daily-workflow-sync, import-workflow, link-checker, and maintainer — all focused on cross-repo workflow management and auto-importing useful workflows. This repo exceeds that level of maturity significantly.

Comparison Summary

This repository exceeds the agentics reference implementation in workflow density and specialization. It matches Pelis Factory practices in security automation, CI diagnosis, and multi-engine testing. It falls short on observability of the agent ecosystem itself and issue lifecycle management.

---

📋 Current Agentic Workflow Inventory

Workflow	Purpose	Trigger	Assessment
security-guard	PR security review	PR opened/sync	✅ Well-configured, AWF-specific context
ci-doctor	CI failure investigation	workflow_run failure	✅ Comprehensive workflow list monitored
doc-maintainer	Documentation sync with code	Daily + PR	✅ Good skip-if-match guard
firewall-issue-dispatcher	Cross-repo issue tracking from gh-aw	Every 6h	✅ Clever cross-repo pattern
dependency-security-monitor	Dependency CVE monitoring	Daily	✅ Creates issues for HIGH/CRITICAL
cli-flag-consistency-checker	CLI flags vs docs consistency	Weekly	✅ Good coverage of doc sources
issue-monster	Issue → Copilot coding agent dispatcher	Issues + hourly	✅ Task dispatcher pattern
update-release-notes	Release notes enhancement	Release published	✅ Nice diff-based automation
test-coverage-improver	Security-critical code test coverage	Weekly	✅ Domain-specific focus on security paths
plan	Planning assistant	/plan slash command	✅ Good interactive pattern
issue-duplication-detector	Duplicate issue detection with cache	Issues opened	✅ Excellent use of cache-memory
security-review	Daily threat modeling	Daily	✅ Comprehensive, evidence-based
ci-cd-gaps-assessment	CI/CD pipeline gap analysis	Daily	⚠️ May overlap with pelis-advisor
build-test	Multi-language build/test	PR opened/sync	✅ Multi-runtime coverage
secret-digger-claude/codex/copilot	Red team secrets search (3 engines)	Hourly	✅ Unique multi-engine comparison
smoke-claude/codex/copilot/chroot	Engine smoke tests	Every 12h + PR	✅ Unique firewall validation
pelis-agent-factory-advisor	This workflow	Weekly	✅ Self-referential meta-workflow

---

🚀 Actionable Recommendations

P0 — Implement Immediately

[P0] Issue Triage Agent

What: Auto-label new issues with categories like bug, feature, documentation, question, security, performance, and add a welcoming comment explaining the label and pointing to relevant docs.

Why: The issue-monster handles assignment but there's no labeling/triage step. Issues arrive unlabeled, making filtering and prioritization harder. In Pelis Factory, the issue triage agent was described as "hello world" — foundational automation that multiplies the value of everything else.

How: Trigger on issues: [opened, reopened]. Read issue content, match against AWF-specific categories (include security label for issues about domain bypass, credential exposure, container escape), add label, comment with next steps. Takes ~5 minutes to implement.

Effort: Low

Example:

---
on:
  issues:
    types: [opened, reopened]
permissions:
  issues: read
tools:
  github:
    toolsets: [issues, labels]
safe-outputs:
  add-labels:
    allowed: [bug, feature, enhancement, documentation, question, security, performance, help-wanted, good-first-issue]
  add-comment:
    max: 1
timeout-minutes: 5
---
# Issue Triage Agent
Analyze new issues in github/gh-aw-firewall. Apply one label from the allowed set.
For security-related issues (bypass, credentials, container escape, iptables, domain filtering),
always apply the `security` label and mention relevant docs in docs/security.md.

---

[P0] Add /fix and /review ChatOps Slash Commands

What: Add slash commands that fire on-demand agents: /fix to attempt automated fixes for labeled bugs, /review to trigger deep security review of a specific PR beyond what security-guard does.

Why: The factory's "Q" agent (78% merge rate, 69 merged PRs) shows slash-command-triggered agents are the highest-ROI ChatOps investment. Currently only /plan exists. This is particularly valuable for a security tool where maintainers want on-demand expert analysis.

How: Add two new slash-command workflows. /fix triggers issue-monster style work on the specific issue. /review triggers a deep security analysis similar to security-review but scoped to the specific PR.

Effort: Low (reuse patterns from plan.md and security-review.md)

---

P1 — Plan for Near-Term

[P1] Audit Workflows (Meta-Agent)

What: A weekly or daily meta-agent that inspects all other agentic workflow runs, analyzes their success/failure rates, token costs, and quality, and posts a discussion with insights. Flags expensive or underperforming workflows.

Why: Pelis Factory's Audit Workflows agent created 93 discussion reports and raised 9 actionable issues. With 22+ workflows running constantly (especially the hourly secret-digger-* trio), there's real risk of cost accumulation and quality degradation without oversight. The ci-cd-gaps-assessment is close but focuses on gaps rather than agent performance.

How: Use tools: agentic-workflows to access run logs. Use tools: cache-memory to track trends. Emit a weekly [Audit] Agent Ecosystem Health discussion.

Effort: Medium

---

[P1] Breaking Change Checker

What: On every PR, analyze if the change introduces backward-incompatible modifications to the AWF public API: CLI flags being removed/renamed, Docker Compose interface changes, environment variable contract changes, or network topology changes.

Why: AWF has a well-defined public interface (CLI flags, env vars, container IPs). Breaking these silently causes integration failures for users. The CI Doctor can catch build failures, but not semantic breaking changes. Pelis Factory's Breaking Change Checker was described as creating alert issues before changes reach production.

How: Trigger on PR, read src/cli.ts diff, compare against documented flags in docs/usage.md and docs-site/, flag removals or renames, create issue or PR comment.

Effort: Medium

---

[P1] Container Image Freshness Monitor

What: Weekly agent that checks if the base images (ubuntu/squid:latest, ubuntu:22.04, Node.js for api-proxy) have known CVEs in their current pinned versions, and proposes updates.

Why: This is a security tool. If its own container images have known vulnerabilities (especially privilege escalation or container escape CVEs), it undermines the entire security model. dependency-security-monitor covers npm packages but not the Docker base images.

How: Use bash to run docker scout or query the GitHub Advisory Database for base image CVEs. Create issues for high/critical findings. Propose Dockerfile updates.

Effort: Medium

---

[P1] Domain Allowlist Advisor

What: A weekly agent that analyzes real Squid access logs from smoke test runs and CI, identifies patterns of blocked-but-legitimate traffic (e.g., npm CDNs, PyPI mirrors), and suggests allowlist improvements for common agent use cases.

Why: AWF is a firewall tool, and its default domain allowlists directly impact the success rate of agents using it. This is uniquely valuable domain intelligence that no generic factory workflow could provide. Real traffic analysis from the smoke tests (smoke-*) would surface gaps in the documented allowlist examples.

How: Read preserved squid logs from CI artifacts, aggregate blocked domains by frequency, filter out known-malicious ones, generate a [Allowlist Advisor] discussion with suggestions for docs/ and examples/.

Effort: Medium

---

P2 — Consider for Roadmap

[P2] Continuous Refactoring Agent

What: Weekly agent that scans src/ for opportunities: functions > 50 lines, duplicate code patterns, TODO/FIXME comments, TypeScript any usages, and proposes PRs to address them.

Why: The codebase has complex modules (docker-manager.ts generates Docker Compose from 500+ line functions). Pelis Factory's Continuous Refactoring created meaningful code quality improvements automatically.

Effort: Medium

---

[P2] Mergefest — Auto-merge Main into Stale PRs

What: Daily agent that finds PRs that are behind main by more than N commits and merges main into them automatically (or creates a comment requesting it).

Why: Long-lived feature PRs in this repo (firewall logic changes) can drift significantly from main, especially with active CI/CD changes. This reduces the "please merge main" friction.

Effort: Low

---

[P2] Issue Arborist — Sub-issue Organizer

What: When multiple related issues are filed (e.g., multiple reports of the same Squid config problem), auto-link them as sub-issues under a parent tracking issue.

Why: Pelis Factory's Issue Arborist created 77 discussion reports and 18 parent issues. For a tool used in many organizations, related issues about the same underlying problem can pile up.

Effort: Low-Medium

---

[P2] Onboarding Checklist Agent

What: When a first-time contributor opens their first PR, post a personalized checklist comment covering AWF-specific requirements: security review considerations, how to run smoke tests locally, the postprocess-smoke-workflows step, etc.

Why: The AGENTS.md and CLAUDE.md files are dense. New contributors frequently miss the post-processing step or don't understand the strict mode compilation requirements. An onboarding comment at PR-open time provides just-in-time education.

Effort: Low

---

P3 — Future Ideas

[P3] Workflow Generator Slash Command

What: A /new-workflow (description) slash command that scaffolds a new .md workflow file, compiles it, and opens a PR — inspired by Pelis Factory's Workflow Generator.

Why: This repo's own workflow generator (if added) could accelerate adding new agentic workflows by generating properly structured templates with correct AWF-specific security settings.

Effort: Medium-High

---

[P3] Protocol Compliance Validator

What: Monthly deep-dive agent that validates the iptables rules, Squid configuration, and container networking actually match what's documented in docs/architecture.md and AGENTS.md. Checks for configuration drift.

Why: Security tools suffer from documentation drift — what's written about how the firewall works vs. what the code actually does. This is particularly risky for the Squid ACL generation and iptables setup.

Effort: High

---

[P3] Performance Regression Agent

What: Extend the existing (non-agentic) performance-monitor.yml with an AI layer that interprets benchmark trends over time and flags regressions with explanations, not just raw numbers.

Why: Container startup time directly impacts UX. An agent that says "startup time increased 40% in the last 3 weeks — likely caused by PR #X which added an extra Docker layer" is more actionable than a raw benchmark number.

Effort: Medium

---

📈 Maturity Assessment

Dimension	Score	Notes
Security Automation	5/5	Exceptional — daily security review, secrets scanning (3 engines!), dependency monitor, security guard
CI/CD Automation	4/5	CI doctor, build-test, smoke tests excellent; missing breaking change checker
Issue Management	3/5	Good assignment (issue-monster), duplication detection; missing triage/labeling
Documentation	4/5	Doc maintainer, CLI checker; good coverage
Meta-observability	2/5	No agent monitoring other agents; biggest gap
ChatOps/Interactive	2/5	Only /plan slash command; large opportunity
Domain-specific	5/5	Firewall issue dispatcher, allowlist patterns in smoke tests; excellent

Current Level: 4/5 — Advanced practitioner. The repository is among the top ~5% of repositories for agentic workflow maturity. The secret-digger trio (3 engines, hourly!) is genuinely novel and impressive.

Target Level: 5/5 — Complete the meta-observability layer (audit workflows), add issue triage, and add 2-3 interactive slash commands.

Gap Analysis: ~3-4 workflows would close the gap to level 5: issue triage, audit meta-agent, and one or two slash commands.

---

🔄 Comparison with Pelis Agent Factory Best Practices

What This Repo Does Well

• Multi-engine testing: Running secret-digger and smoke tests across Claude, Codex, and Copilot simultaneously is beyond what even the factory does — it's unique validation that AWF works as an engine-agnostic firewall
• Security depth: Daily threat modeling + security guard on PRs + dependency monitoring + secrets scanning = defense in depth applied to the workflow layer itself
• Cache-memory usage: issue-duplication-detector uses cache-memory correctly for persistence — exactly the pattern recommended by the factory
• Cross-repo automation: firewall-issue-dispatcher bridges gh-aw and gh-aw-firewall issues — a sophisticated cross-repo pattern
• Strict mode: Using AWF to sandbox its own workflows (dogfooding!) is meta and impressive
• Domain-specific context: Workflows like security-review.md and security-guard.md include deep AWF-specific context about container escape, iptables manipulation, and Squid ACLs

What Could Be Improved

• Agent ecosystem visibility: No meta-agent watching the agents. With hourly workflows (secret-digger runs every hour × 3 engines = 72 runs/day), cost and quality tracking is essential
• Issue lifecycle completeness: The missing triage link means issues arrive without labels, making the issue-monster's assignment decisions less informed
• Interactive patterns: The factory learned that "context is king" for slash commands — being able to summon an expert agent at the exact moment you need it (on a specific PR or issue) is high-ROI

---

📝 Notes for Future Runs

Stored in /tmp/gh-aw/cache-memory/advisor-notes.json:

• 22 agentic workflows found as of 2026-03-29
• Top gaps: issue triage, meta-audit, breaking change checker, allowlist advisor
• Maturity level 4/5
• Next run should check if issue-triage-agent was added and if audit-workflows meta-agent was implemented

AI generated by Pelis Agent Factory Advisor

• expires on Apr 5, 2026, 3:42 AM UTC