Add autonomous maintenance daemon with source monitoring and triage dispatch

[maxine-at-forecast]

Summary

Build an autonomous maintenance daemon for crosslink that monitors external sources (GitHub issues, CI workflows, dependency releases, internal dev chatter, crosslink issues) and autonomously dispatches low-hanging-fruit tasks — with a human filter layer for anything beyond trivial.

This extends the spirit of the /maintain slash command (which runs a one-shot audit pass) into a persistent, event-driven system that continuously triages and acts.

Motivation

The /maintain skill already demonstrates the value of structured codebase maintenance (dependency audit, lint health, dead code, issue hygiene). But it's manual — someone has to remember to run it. Meanwhile, maintenance signals arrive continuously from many sources:

• GitHub issues: bug reports, feature requests, community feedback
• CI/CD: flaky tests, build failures, security alerts, Dependabot PRs
• Dependency ecosystem: new releases, yanked crates, security advisories
• Crosslink issues: stale issues, orphaned subissues, unlabeled work
• Dev chatter: Slack/Discord mentions, PR review comments

Today these pile up and get batch-processed during periodic human triage. A daemon that watches these sources and autonomously handles the unambiguous cases (while flagging the rest for human review) would dramatically reduce maintenance toil.

Architecture Questions

Should this be a separate crate?

crosslink is already substantial (~102k lines, 153 files). The daemon would add:

• Source adapters (GitHub API polling, webhook receivers, RSS/Atom for releases)
• A rule engine for triage policies
• Agent dispatch logic (spawning kickoff agents for approved tasks)
• State management (what's been seen, what's in-flight, what's waiting for human review)

Options:

1. Separate crate in the workspace (crosslink-maintain or crosslink-sentinel): clean boundary, own dependency tree, can evolve independently. Imports crosslink as a library for DB/issue/kickoff APIs.
2. Module within crosslink: simpler build, shared types directly, but adds to the monolith.
3. Separate binary that uses crosslink as a library: strongest separation, but requires stabilizing crosslink's internal APIs as a public interface.

Leaning toward (1) — workspace crate with a clear API boundary.

Level of generality

The daemon needs a plugin/adapter architecture for sources and actions. Key question: how generic should the rule engine be?

• Minimal: hardcoded source→action mappings (e.g., "GH issue with agent-todo: replicate → spawn replication agent"). Fast to build, easy to reason about.
• Policy-based: declarative rules in a config file (e.g., YAML/TOML policies like "when source=github AND label=agent-todo:replicate, dispatch=kickoff-replicate"). More flexible, but more complex.
• Full workflow engine: arbitrary DAGs of condition→action steps. Maximum power, maximum complexity.

Suggest starting minimal with an eye toward policy-based as V2.

Concrete Test Case: agent-todo: replicate Dispatch

Current manual flow

1. User files a GH issue describing a bug or behavior
2. Human triages and adds agent-todo: replicate label as a signal that this is safe for an agent to attempt
3. (Today: nothing happens automatically — someone has to notice the label and act)

Proposed automated flow

1. Daemon polls GH issues (or receives webhook) for label changes
2. Detects agent-todo: replicate was added to GH issue #N
3. Creates a crosslink issue linked to GH #N
4. Spawns a kickoff agent scoped to: "Reproduce the bug described in GH #N, write a failing test, and report findings"
5. Agent runs in a sandboxed worktree with limited scope (read-only except for test files)
6. On completion, daemon posts results back to the GH issue as a comment
7. Human reviews the reproduction and decides next steps

This is a good first target because:

• The human filter layer (agent-todo: label) is already in practice
• The task is well-scoped (write a failing test, not fix the bug)
• Success/failure is mechanically verifiable (test fails = reproduced)
• Low blast radius (only touches test files)

Meta-Cognitive Flows for Software Maintenance

Beyond the concrete dispatch case, we should think about what maintenance patterns the daemon should eventually support:

Reactive flows (respond to external events)

• Bug triage: GH issue → classify severity → attempt reproduction → report
• Dependency alert: advisory published → check if affected → file issue or auto-patch
• CI failure: build breaks on main → bisect → file issue with root cause
• Stale PR: PR open >N days with no activity → ping author or close

Proactive flows (periodic sweeps)

• Lint drift: run clippy/eslint weekly → fix new warnings → PR
• Test coverage: measure coverage delta → identify uncovered new code → write tests
• Doc freshness: check README/CHANGELOG against recent commits → flag stale sections
• Issue hygiene: close stale issues, merge duplicates, add missing labels

Learning flows (improve over time)

• Track dispatch outcomes: which agent-todo tasks succeeded vs failed?
• Tune thresholds: adjust what counts as "low-hanging fruit" based on historical success rate
• Surface patterns: "tests in module X break 3x more often than average" → file structural issue

Relationship to Existing Code

• daemon.rs: Current daemon is a simple flush-interval loop for hydration. The maintenance daemon would either extend this or run as a separate process.
• /maintain skill (resources/claude/commands/maintain.md): One-shot audit. The daemon's proactive flows are essentially /maintain sections running on a schedule.
• crosslink kickoff: The dispatch mechanism — daemon spawns agents via kickoff for approved tasks.
• GH Future: kickoff workflow expansions & VDD-IAR integration #23 (kickoff workflow expansions): Related future work on multi-agent coordination and remote execution.
• GH Epic: Codebase maintainability audit — structural refactors for long-term health #498 (maintainability audit): Structural refactors that would make the codebase easier for the daemon to maintain.

Open Questions

• What should the daemon be called? (crosslink sentinel, crosslink patrol, crosslink maintain daemon, etc.)
• Should source adapters be compiled-in or dynamically loaded (WASM plugins, shared libs)?
• How should the human filter layer work beyond labels? (Slack approval buttons? TUI approval queue? Web dashboard?)
• What's the trust model for daemon-spawned agents? Should they have reduced permissions by default?
• How does this interact with the existing crosslink trust system?

Suggested Milestones

1. V0: Architecture design doc + separate crate skeleton + GH webhook adapter
2. V1: agent-todo: replicate end-to-end flow (poll → dispatch → report)
3. V2: Policy engine + 2-3 more source adapters (CI, dependencies)
4. V3: Proactive flows (scheduled maintenance sweeps)
5. V4: Learning flows (outcome tracking, threshold tuning)

[dollspace-gay]

Design Document: Autonomous Maintenance Daemon (crosslink patrol)

1. Overview

A persistent command that monitors external sources and autonomously dispatches scoped maintenance tasks via the existing kickoff infrastructure. Starts minimal (one flow, one source) and grows toward a multi-source, policy-driven system.

Name: crosslink patrol (subcommand group)

• crosslink patrol run — one-shot sweep (like /maintain but with dispatch)
• crosslink patrol watch — persistent daemon mode with configurable interval
• crosslink patrol status — show daemon status and recent dispatch history
• crosslink patrol history — show past patrol runs and outcomes

2. Architecture

2.1 Where It Lives

Phase 1 (V0–V1): New module at commands/patrol/ within the existing crosslink crate. This avoids premature crate extraction and gives direct access to Database, SharedWriter, KickoffOpts, and SyncManager without stabilizing a public API boundary.

Phase 2 (V2+): Extract to crosslink-patrol workspace crate when the module exceeds ~2000 lines or needs its own dependency tree (e.g., webhook server). At that point the internal API boundary will be obvious from actual usage.

crosslink/src/commands/patrol/
├── mod.rs          # CLI dispatch + PatrolCommands enum
├── engine.rs       # Core patrol loop (one-shot and watch modes)
├── sources/
│   ├── mod.rs      # Source trait + registry
│   ├── github.rs   # GH issue/label/CI polling via `gh` CLI
│   └── internal.rs # Crosslink issue hygiene checks
├── dispatch.rs     # Triage rules + kickoff agent spawning
├── history.rs      # Outcome recording + reporting
└── config.rs       # Patrol-specific configuration

2.2 Core Abstractions

/// A maintenance signal detected by a source adapter.
pub struct Signal {
    pub source: SourceKind,       // GitHub, CI, Internal, etc.
    pub kind: SignalKind,         // NewIssue, LabelAdded, CIFailure, etc.
    pub reference: String,        // e.g. "GH#499", "CI:run/12345"
    pub title: String,
    pub body: String,
    pub metadata: serde_json::Value,
    pub detected_at: DateTime<Utc>,
}

/// What the triage engine decides to do with a signal.
pub enum Disposition {
    /// Spawn a kickoff agent with this scope.
    Dispatch { description: String, scope: AgentScope },
    /// Create a crosslink issue for human review.
    Triage { priority: String, labels: Vec<String> },
    /// Already handled or duplicate — skip.
    Skip { reason: String },
}

/// Constrains what a dispatched agent can do.
pub struct AgentScope {
    pub allowed_paths: Vec<String>,   // e.g. ["tests/", "crosslink/tests/"]
    pub verify: VerifyLevel,
    pub timeout: Duration,
    pub model: String,
}

2.3 Source Trait

pub trait Source {
    /// Human-readable name for logging.
    fn name(&self) -> &str;

    /// Poll for new signals since last check.
    /// Returns signals that haven't been seen before.
    fn poll(&mut self, seen: &SeenSet) -> Result<Vec<Signal>>;
}

Phase 1 sources (V0–V1):

• GitHubLabelSource — polls for issues with agent-todo:* labels via gh issue list --label "agent-todo:replicate" --json number,title,body,labels
• InternalHygieneSource — checks crosslink issues for staleness, orphaned subissues, missing labels

Phase 2+ sources (when needed):

• GitHubCISource — polls gh run list for failures on default branch
• DependencySource — cargo audit / npm audit periodic checks
• WebhookSource — HTTP listener for real-time GH webhooks (requires axum, already a dependency)

2.4 Triage Rules (Hardcoded V1, Policy V2)

fn triage(signal: &Signal) -> Disposition {
    match (&signal.source, &signal.kind) {
        // agent-todo: replicate → spawn reproduction agent
        (SourceKind::GitHub, SignalKind::LabelAdded)
            if signal.metadata["label"] == "agent-todo: replicate" =>
        {
            Disposition::Dispatch {
                description: format!(
                    "Reproduce bug from GH#{}: {}",
                    signal.reference, signal.title
                ),
                scope: AgentScope {
                    allowed_paths: vec!["tests/".into()],
                    verify: VerifyLevel::Local,
                    timeout: Duration::from_secs(1800),
                    model: "claude-sonnet-4-6".into(),
                },
            }
        }

        // Stale crosslink issue (>30 days, no activity) → triage for human
        (SourceKind::Internal, SignalKind::StaleIssue) => {
            Disposition::Triage {
                priority: "low".into(),
                labels: vec!["hygiene".into()],
            }
        }

        _ => Disposition::Skip {
            reason: "no matching rule".into(),
        },
    }
}

V2 policy format (only when 3+ hardcoded rules exist and patterns emerge):

# .crosslink/patrol.toml
[[rule]]
name = "replicate-bugs"
source = "github"
match = { label = "agent-todo: replicate" }
action = "dispatch"
scope = { paths = ["tests/"], timeout = "30m", model = "claude-sonnet-4-6" }

[[rule]]
name = "stale-issues"
source = "internal"
match = { kind = "stale", days = 30 }
action = "triage"
priority = "low"
labels = ["hygiene"]

3. Dispatch Flow (V0 Target: agent-todo: replicate)

┌──────────────┐     ┌─────────────┐     ┌──────────────┐     ┌────────────┐
│ GH Issue     │     │ Patrol      │     │ Kickoff      │     │ Agent      │
│ + label      │────▶│ Engine      │────▶│ Run          │────▶│ (tmux)     │
│ agent-todo:  │     │ (triage)    │     │ (worktree)   │     │            │
│ replicate    │     └──────┬──────┘     └──────────────┘     └─────┬──────┘
└──────────────┘            │                                       │
                            │                                       │
                    ┌───────▼───────┐                        ┌──────▼──────┐
                    │ History DB    │◀───────────────────────│ Results     │
                    │ (patrol runs) │     (on completion)    │ (.kickoff-  │
                    └───────┬───────┘                        │  status)    │
                            │                                └─────────────┘
                    ┌───────▼───────┐
                    │ GH Comment    │
                    │ (results)     │
                    └───────────────┘

Step-by-step:

1. Poll: gh issue list --repo <repo> --label "agent-todo: replicate" --json number,title,body,labels,createdAt
2. Dedup: Check patrol_history table — skip issues already dispatched
3. Create crosslink issue: db.create_issue(...) linked to GH issue number
4. Spawn agent: kickoff::run(crosslink_dir, db, writer, &opts) with a scoped prompt:

   Reproduce the bug described in GH#<N>.
   
   Issue title: <title>
   Issue body: <body>
   
   Your task:
   1. Read the issue carefully
   2. Write a failing test that demonstrates the bug
   3. Run the test to confirm it fails
   4. Record your findings via crosslink comments
   
   You may ONLY modify files in tests/.
   

5. Monitor: Patrol checks .kickoff-status in the worktree (existing sentinel)
6. Report: On completion, post results to GH issue via gh issue comment <N> --body "<results>"
7. Record: Write outcome to patrol_history table

4. State Management

4.1 Patrol History Table (SQLite)

Added to the existing issues.db:

CREATE TABLE IF NOT EXISTS patrol_runs (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    run_id TEXT NOT NULL,           -- UUID for this patrol run
    started_at TEXT NOT NULL,
    completed_at TEXT,
    mode TEXT NOT NULL,             -- "oneshot" or "watch"
    signals_found INTEGER DEFAULT 0,
    dispatched INTEGER DEFAULT 0,
    triaged INTEGER DEFAULT 0,
    skipped INTEGER DEFAULT 0
);

CREATE TABLE IF NOT EXISTS patrol_dispatches (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    run_id TEXT NOT NULL,
    signal_ref TEXT NOT NULL,       -- "GH#499"
    source TEXT NOT NULL,           -- "github"
    disposition TEXT NOT NULL,      -- "dispatch", "triage", "skip"
    agent_id TEXT,                  -- kickoff agent ID (if dispatched)
    crosslink_issue_id INTEGER,    -- local crosslink issue (if created)
    outcome TEXT,                   -- "success", "failure", "timeout", "pending"
    outcome_detail TEXT,           -- agent's findings or error message
    created_at TEXT NOT NULL,
    completed_at TEXT,
    FOREIGN KEY (crosslink_issue_id) REFERENCES issues(id)
);

4.2 Seen Set

/// Tracks which signals have already been processed to avoid re-dispatch.
struct SeenSet {
    /// signal_ref → disposition
    seen: HashMap<String, String>,
}

impl SeenSet {
    fn load(db: &Database) -> Result<Self> {
        // SELECT signal_ref, disposition FROM patrol_dispatches
    }

    fn contains(&self, reference: &str) -> bool {
        self.seen.contains_key(reference)
    }
}

5. Daemon Mode (patrol watch)

Extends the existing daemon.rs pattern:

pub fn watch(crosslink_dir: &Path, interval: Duration) -> Result<()> {
    // Write PID file (like existing daemon)
    // Register signal handlers (SIGTERM, SIGINT)
    
    let db = Database::open(&crosslink_dir.join("issues.db"))?;
    let mut sources: Vec<Box<dyn Source>> = vec![
        Box::new(GitHubLabelSource::new()?),
        Box::new(InternalHygieneSource::new(crosslink_dir)),
    ];
    
    loop {
        if should_exit.load(Ordering::Relaxed) {
            break;
        }
        
        // One patrol cycle
        if let Err(e) = run_patrol_cycle(&db, &mut sources, crosslink_dir) {
            tracing::error!("patrol cycle failed: {}", e);
            consecutive_failures += 1;
            if consecutive_failures > 5 {
                tracing::warn!("5 consecutive failures — backing off");
            }
        } else {
            consecutive_failures = 0;
        }
        
        // Sleep with early wake on signal
        std::thread::sleep(interval);
    }
}

Key difference from existing daemon: The hydration daemon runs every 30s for lightweight state sync. Patrol runs every 5–15 minutes (configurable) and may spawn agents that run for 30+ minutes. The patrol loop must track in-flight agents and collect results asynchronously.

6. Trust Model for Daemon-Spawned Agents

Leverages the work from #505 (auto-trust for kickoff subagents):

• Patrol-spawned agents use the same init_worktree_agent path as kickoff
• They get dedicated signing keys and are auto-approved
• Additional constraint: patrol agents should default to VerifyLevel::Local (no push/PR creation) unless the rule explicitly escalates
• Agent worktrees are cleaned up automatically after result collection

7. Configuration

// .crosslink/hook-config.json additions
{
    "patrol": {
        "enabled": true,
        "interval_minutes": 10,
        "max_concurrent_agents": 3,
        "sources": {
            "github_labels": {
                "enabled": true,
                "labels": ["agent-todo: replicate", "agent-todo: fix"]
            },
            "internal_hygiene": {
                "enabled": true,
                "stale_threshold_days": 30
            }
        },
        "default_agent": {
            "model": "claude-sonnet-4-6",
            "timeout_minutes": 30,
            "verify": "local"
        }
    }
}

8. Human Filter Layer

Three tiers of autonomy:

Tier	When	Human involvement
Auto-dispatch	Signal matches a dispatch rule AND has an agent-todo: label (human already triaged)	None — agent runs, results posted
Triage-and-wait	Signal matches a triage rule	Crosslink issue created, human decides
Manual approval	Future: Slack/webhook integration	Approval button before dispatch

The agent-todo: label convention is the existing human filter — the daemon doesn't invent new autonomy, it automates the response to signals that humans have already blessed.

9. Milestones

V0: Foundation (this PR)

• commands/patrol/ module skeleton
• patrol_runs + patrol_dispatches tables
• GitHubLabelSource — poll agent-todo: replicate issues
• Dispatch via kickoff::run with scoped reproduction prompt
• Result collection from .kickoff-status + agent comments
• Post results back to GH issue via gh issue comment
• crosslink patrol run (one-shot) working end-to-end
• crosslink patrol history (show past runs)

V1: Daemon Mode

• crosslink patrol watch with configurable interval
• In-flight agent tracking (don't re-dispatch while agent is running)
• crosslink patrol status (live dashboard)
• Concurrent agent limit enforcement
• agent-todo: fix dispatch rule (beyond just replication)

V2: Multi-Source + Policy

• InternalHygieneSource (stale issues, orphans, missing labels)
• GitHubCISource (build failures on default branch)
• patrol.toml policy file (declarative rules)
• DependencySource (cargo audit / npm audit)
• Outcome tracking + success rate metrics

V3: Proactive + Learning

• Scheduled maintenance sweeps (lint drift, test coverage)
• Webhook receiver for real-time GH events (axum endpoint)
• Historical pattern detection ("module X breaks 3x more than average")
• Threshold tuning based on dispatch outcomes

10. Open Questions for Team Input

1. Naming: Is patrol the right name? Alternatives: sentinel, watch, maintain-daemon
2. First label beyond replicate: Should agent-todo: fix auto-dispatch, or is reproduction-only the right conservative starting point?
3. Result posting format: Structured markdown template, or free-form agent output?
4. Slack integration timeline: Is this V1 (approval buttons) or V2+ (nice-to-have)?
5. Model selection: Should patrol agents default to Sonnet (cheaper, faster) or Opus (more capable)? Per-rule override?

[m13v]

the human filter layer is the part that makes or breaks this. we built a similar autonomous system for scheduled social posting - agents that run on launchd timers, scan for relevant threads, draft content, and post. the original version had no filter and it would occasionally do something dumb. what we settled on is a tiered risk model - tier 1 actions (read-only, scan, triage) just run. tier 2 (create branch, draft PR) run but notify the human. tier 3 (merge, deploy, anything destructive) queue for explicit approval. the dispatch part is straightforward, it's calibrating which tier each action falls into that took the most iteration. also, deduplication is critical - if the daemon fires every 4 hours and the same GitHub issue is still open, you need to track that you already triaged it so you don't create duplicate work items.

[m13v]

here's the launchd scheduling setup we use for our autonomous daemon runs - plist configs, interval management, and the dedup tracking: https://github.com/m13v/social-autoposter/tree/main/launchd