[Pelis Agent Factory Advisor] Agentic Workflow Maturity Report – March 2026

[github-actions[bot]]

📊 Executive Summary

This repository has an impressive and mature agentic workflow setup with 21 compiled workflows covering security, testing, documentation, CI/CD, and issue management — far above most repositories. The top opportunities center on issue triage automation (currently missing a basic triage agent), PR-fix ChatOps, workflow health meta-monitoring, and domain-specific firewall validation workflows that would leverage the unique security expertise of this codebase.

---

🎓 Patterns Learned from Pelis Agent Factory

Key Patterns from the Documentation Site

Pattern	Description	Status Here
Issue Triage Agent	Auto-label issues on open; foundational "hello world" workflow	❌ Missing
Skip-if-match guard	Prevent parallel runs via skip-if-match frontmatter	✅ Used in issue-monster, doc-maintainer
Cache-memory state	Persist state across daily runs for multi-phase work	✅ Used in issue-duplication-detector
Meta-agent (Workflow Health Manager)	Agent that monitors other agents' runs	❌ Missing
PR Fix Slash Command	On-demand /pr-fix to fix failing CI	❌ Missing
Changeset / Version Automation	Auto semver bump from commit analysis	⚠️ Partial (update-release-notes exists but no semver bump)
Daily Malicious Code Scan	Scan recent commits for suspicious patterns	❌ Missing
Breaking Change Checker	Detect backward-incompatible changes on PRs	❌ Missing

Key Patterns from the Agentics Repository

The githubnext/agentics repository contributes these notable patterns:

• daily-test-improver – Incremental test gap filling, one area per day
• daily-backlog-burner – Work through issues systematically, one per day
• pr-nitpick-reviewer – Thorough quality gating before merge
• grumpy-reviewer – Adversarial review perspective
• contribution-guidelines-checker – Validates PRs against CONTRIBUTING.md

Comparison with Current Implementation

This repository already applies many advanced patterns: multi-engine smoke tests (Claude, Codex, Copilot), hourly red-team secret scanning with 3 engines, cache-memory for persistence, shared workflow imports, and domain-specific security guards. The main gap versus the factory is the absence of issue management/triage and meta-observability workflows.

---

📋 Current Agentic Workflow Inventory

Workflow	Purpose	Trigger	Assessment
build-test	Build/test on PRs	PR open/sync	✅ Well-designed
ci-cd-gaps-assessment	CI/CD gap analysis	Daily	✅ Good observability
ci-doctor	Investigate CI failures	workflow_run failure	✅ Strong pattern, 69% merge rate in gh-aw
cli-flag-consistency-checker	CLI flag doc sync	Weekly	✅ Solid
dependency-security-monitor	Dependency CVE monitor	Daily	✅ Comprehensive
doc-maintainer	Docs sync with code	Daily	✅ Good with skip guard
issue-duplication-detector	Dedup issues	Issue opened	✅ Good cache-memory usage
issue-monster	Dispatch issues to Copilot	Hourly + issue open	✅ Well-guarded
pelis-agent-factory-advisor	This workflow	Daily	✅ Meta pattern
plan	/plan slash command	Slash command	✅ ChatOps pattern
secret-digger-claude/codex/copilot	Red team secret scans	Hourly (3 engines)	✅ Excellent multi-engine coverage
security-guard	PR security review	PR open/sync	✅ Claude engine, security-focused
security-review	Daily threat modeling	Daily	✅ Thorough
smoke-chroot/claude/codex/copilot	End-to-end smoke tests	PR + scheduled	✅ Exceptional multi-engine coverage
test-coverage-improver	Improve test coverage	Weekly	✅ PR-generating, skip-guarded
update-release-notes	Enhance release notes	Release published	✅ Good automation

Coverage: 38.39% overall. cli.ts at 0%, docker-manager.ts at 18% — major gap in core code.

---

🚀 Actionable Recommendations

P0 – Implement Immediately

---

[P0] Issue Triage Agent

What: Automatically label incoming issues (bug, feature, enhancement, documentation, question, security, help-wanted) when they are opened.

Why: This repository has an active issue tracker but zero automatic labeling. The Pelis Factory's most-cited "hello world" workflow — simple to build, immediate value. The issue-monster dispatcher would also benefit from pre-labeled issues.

How: Add issue-triage-agent.md triggered on issues: [opened, reopened] using GitHub issues toolset + safe-outputs: add-labels. Given this is a public security tool, keep lockdown: true (default).

Effort: Low (< 1 hour to add and compile)

Example:

---
name: Issue Triage Agent
on:
  issues:
    types: [opened, reopened]
permissions:
  issues: read
tools:
  github:
    toolsets: [issues, labels]
safe-outputs:
  add-labels:
    allowed: [bug, feature, enhancement, documentation, question, security, help-wanted, good-first-issue]
  add-comment: {}
timeout-minutes: 5
---
Analyze issue #$\{\{ github.event.issue.number }} and apply the most appropriate label.
Research the issue in context of this firewall/security tool codebase.

---

[P0] Daily Firewall Domain Audit (Domain-Specific)

What: A workflow that validates the actual domain allowlist/blocklist enforcement by running awf smoke test invocations and reporting which domains are correctly allowed/blocked.

Why: This is the core security product — domain filtering. A daily automated audit using the built-in awf logs stats and test harness would catch regressions in domain ACL logic that existing unit tests miss. This is unique to this repo and would directly validate the product's primary security guarantee.

How: Triggered daily, runs awf --allow-domains example.com -- curl https://github.com and validates the access log to confirm github.com was blocked. Creates a discussion with firewall validation results.

Effort: Medium (2-3 hours, requires understanding the test fixtures)

---

P1 – Plan for Near-Term

---

[P1] PR Fix Slash Command

What: A /pr-fix slash command that, when invoked in a PR comment, analyzes failing CI checks and attempts to automatically fix them.

Why: The Pelis Factory's pr-fix workflow has high utility — when CI fails on a PR, the author can type /pr-fix instead of debugging manually. Especially valuable for the complex build pipeline here (TypeScript compilation + container tests + lint).

How: slash_command: pr-fix in workflow, reads the failing job logs via GitHub Actions tools, proposes a PR fix.

Effort: Low-Medium

---

[P1] Workflow Health Manager (Meta-Agent)

What: A daily meta-agent that monitors the health of all other agentic workflows — checking for failures, cost anomalies, workflows that haven't run, and degraded output quality.

Why: With 21 workflows running, a meta-observer is essential. In the Pelis Factory, this pattern created 40 issues and 34 merged PRs. It catches workflows that are silently failing or producing low-quality output.

How: Daily schedule, uses agentic-workflows MCP to audit recent runs, cache-memory to track trends, creates issues for degraded workflows.

Effort: Medium

---

[P1] Breaking Change Checker

What: On every PR, detect CLI flag removals, API breaking changes, and Docker Compose config changes that would break existing users.

Why: This is a CLI tool with external users. Breaking changes to --allow-domains, --build-local, etc. are high-impact. The Pelis Factory's equivalent created alert issues that caught production-breaking changes. Given the security implications here (misconfigured firewall = security bypass), catching these early is critical.

How: pull_request trigger, checks for removed CLI options, changed Docker network config, modified iptables rules. Creates a comment on the PR if breaking changes detected.

Effort: Low-Medium

---

[P1] Automated Changeset / Version Bump

What: When a PR is merged to main, automatically analyze commits and create a PR proposing the appropriate semantic version bump and CHANGELOG entry.

Why: The update-release-notes workflow runs on release: published, but there's no automation to determine the right version bump. The Pelis Factory's changeset workflow had a 78% merge rate (22/28 PRs merged). This reduces the "what version should this be?" decision friction.

How: Triggered on push to main, analyzes commits for feat:, fix:, BREAKING CHANGE: markers, proposes semver bump in a PR.

Effort: Medium

---

P2 – Consider for Roadmap

---

[P2] Daily Malicious Code Scan

What: Daily review of commits from the past 24 hours for suspicious code patterns (credential harvesting, backdoors, obfuscated code, suspicious network calls).

Why: This is a security tool with an npm package and Docker images. Supply chain attacks are a real threat. The Pelis Factory runs this daily. Given this repo protects AI agent network access, any compromise would be high-impact.

How: Daily schedule, uses git log --since="24 hours ago" + code review of changed files.

Effort: Low (can remix from gh-aw factory)

---

[P2] PR Nitpick Reviewer

What: A thorough quality reviewer that checks PRs for: missing tests, documentation gaps, error handling gaps, and TypeScript type safety issues.

Why: The existing security-guard focuses on security; a separate quality reviewer would catch non-security issues. The Pelis Factory's equivalent had consistent quality improvements.

How: pull_request trigger, separate from security-guard, uses bash tools to run lint/type-check.

Effort: Low

---

[P2] Weekly Repository Chronicle

What: Weekly summary of repository activity — PRs merged, issues opened/closed, workflow success rates, coverage trends.

Why: Keeps maintainers informed of project velocity without manual tracking. The daily-repo-chronicle in agentics created useful weekly status reports.

How: Weekly schedule, queries GitHub API for weekly activity, posts a discussion summary.

Effort: Low

---

[P2] Documentation Noob Tester

What: Periodically test the documentation from a "first-time user" perspective — can a new user follow the Quick Start guide? Are the examples accurate?

Why: The docs site (docs-site/) is Astro/Starlight, and the repo has extensive documentation. The Pelis Factory's noob tester had a 43% merge rate but surfaced real usability issues. For a security tool, documentation accuracy directly impacts secure usage.

How: Weekly schedule, simulates following Quick Start instructions, tests code examples.

Effort: Medium (requires bash tools to run commands)

---

[P2] GitHub Actions Updater

What: Weekly check for outdated action pins (SHA-pinned actions that have newer versions).

Why: The repo uses SHA-pinned actions (security best practice) but those pins go stale. The Pelis Factory's daily-workflow-updater kept actions current. Currently dependency-security-monitor covers npm/dependencies but not action pins.

How: Weekly schedule, checks action versions, creates PR with updated SHA pins.

Effort: Low-Medium

---

P3 – Future Ideas

---

[P3] Issue Arborist

What: Periodically link related issues as parent/sub-issues, building a dependency tree.

Why: With active issue tracking and complex interrelated features, organizing issues into trees would help with project planning. The Pelis Factory's arborist created 18 parent issues grouping related work.

Effort: Medium

---

[P3] Repo-Ask ChatOps

What: A /ask slash command that answers questions about the codebase — "How does the HTTPS proxy work?", "What domains are allowlisted by default?".

Why: The codebase has complex architecture (three containers, iptables, Squid). A ChatOps Q&A would reduce friction for contributors and users. The repo-ask pattern from agentics is a good template.

Effort: Low

---

[P3] Container Drift Detector

What: Daily comparison of container security configurations against a known-good baseline.

Why: This is the most unique opportunity for this repo. Drift in seccomp-profile.json, capability dropping, or iptables setup could silently weaken the firewall. No other repo in the factory has this — it would be a novel contribution.

Effort: Medium

---

[P3] Grumpy Reviewer

What: An adversarial PR reviewer that specifically tries to find security weaknesses and challenges assumptions.

Why: The security-guard is constructive; a "grumpy" variant would be more adversarial and challenge design decisions. Given the security domain, an adversarial reviewer adds unique value.

Effort: Low

---

📈 Maturity Assessment

Dimension	Score	Notes
Security automation	⭐⭐⭐⭐⭐	Best-in-class: 3 secret scanners, security-guard, security-review, dependency-monitor
Testing automation	⭐⭐⭐⭐	Multi-engine smoke tests, coverage improver, but 38% overall coverage
Documentation automation	⭐⭐⭐	doc-maintainer runs daily, but no noob-tester or unbloat
Issue management	⭐⭐	issue-monster + dedup detector, but no triage/labeling
Release automation	⭐⭐⭐	Release notes, but no semver automation
Observability/Meta	⭐⭐	CI doctor + gaps assessment, but no workflow health manager
ChatOps	⭐⭐	/plan command only

Current Level: 3.5/5 — Significantly above average, with exceptional security-specific coverage. Gaps are primarily in issue management basics and meta-observability.

Target Level: 4.5/5 — Achievable by adding issue triage (P0), PR-fix slash command (P1), workflow health manager (P1), and 2-3 P2 items.

Gap: ~5-7 new workflows needed to reach target level.

---

🔄 Comparison with Pelis Agent Factory Best Practices

What This Repo Does Well

• Multi-engine coverage: Running smoke tests across Claude, Codex, and Copilot simultaneously is exemplary
• Security depth: 3 secret diggers + security-guard + daily security review far exceeds factory standards
• Shared imports: Clean use of shared/ workflow fragments for pagination, reporting, etc.
• Skip guards: skip-if-match properly prevents parallel run conflicts
• Cache-memory: Good use in issue-duplication-detector

What It Could Improve

• Issue lifecycle automation: Missing triage labeling (foundational pattern in factory)
• Meta-observability: No workflow health manager watching the watchers
• ChatOps breadth: Only /plan; could add /pr-fix, /ask
• Incremental multi-phase: No daily-backlog-burner style multi-day campaigns

Unique Opportunities (Domain: Security/Firewall)

The factory has no firewall/network security tool workflows. This repo could pioneer:

• Domain ACL Regression Tests as a first-class agentic workflow
• Container Security Drift Detection as a novel pattern
• Adversarial Firewall Testing where an agent tries to bypass its own firewall rules

---

Analysis based on 21 existing agentic workflows and patterns from Pelis Agent Factory (19-part series) and githubnext/agentics repository. Cache-memory updated at /tmp/gh-aw/cache-memory/pelis-advisor-notes.json.

AI generated by Pelis Agent Factory Advisor

• expires on Mar 27, 2026, 3:26 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-27T03:26:02.059Z.

Closed by Workflow