-
Notifications
You must be signed in to change notification settings - Fork 6
Description
Problem
The system has no feedback loops between phases. The orchestration is fire-and-forget:
- No recon-informed prioritization: If recon finds a critical misconfiguration (exposed .env), all 12 hunt strategies still run at same depth instead of fast-tracking config-related hunters.
- No signal density adaptation: If 8/12 hunters return zero findings, no adaptive behavior — no depth escalation, no fallback to broader scanning.
- No prove-to-hunt feedback: If prove demotes 80% of findings, hunt does not re-calibrate or explore additional angles.
- Single decision point: The AI gate for strategy selection is a one-shot decision, not continuous adaptation.
Architectural Principle Violated
Principle 2 (Guided Autonomy): Agents should respond to intermediate results.
Principle 3 (Dynamic Orchestration): Output of one agent determines structure of the subsequent sub-system.
Proposed Design: Signal-Based Adaptive Orchestrator
Core Concept: Orchestration Signals
class OrchestrationSignal(BaseModel):
signal_type: str # "finding_density", "critical_finding", "zero_results", "budget_alert"
source_phase: str
source_agent: str
payload: dict[str, Any]Adaptation Rules
Rule 1: Recon-Informed Hunt Prioritization
When recon finds critical config issues, boost config_secrets hunter priority and depth.
Rule 2: Signal Density to Depth Escalation
After first batch of hunters, if findings_per_hunter < 0.5 and depth == "standard", escalate remaining hunters to "thorough".
Rule 3: Prove Demotion to Hunt Re-scan
If prove demotes >60% of findings from a strategy, re-run that hunter with tightened confidence threshold (only high-confidence findings).
Rule 4: Budget-Aware Throttling
If budget >70% consumed, reduce prove concurrency, only verify critical/high severity.
Where This Fits in the Reasoner DAG
The adaptive controller is orchestration logic, NOT a reasoner. It observes signals and modifies parameters passed to reasoners. Reasoners remain pure.
Implementation Strategy
Phase 1: Signal infrastructure + Rule 2 (density adaptation)
Phase 2: Rule 1 (recon-informed prioritization)
Phase 3: Rule 3 (prove-to-hunt feedback loop)
Phase 4: Rule 4 (budget throttling — partial exists already)
Acceptance Criteria
- Signal infrastructure: OrchestrationSignal model + queue
- Rule 2: Low signal density triggers depth escalation
- Rule 1: Recon critical findings prioritize relevant hunters
- Metrics: Log adaptation decisions for observability
- No regression in finding quality (DVGA benchmark)
- Backward compatible: adaptive behavior default-on with override
Dependencies
- Benefits from streaming pipeline (arch: streaming phase pipeline — eliminate sequential recon→hunt→prove blocking #105) — streaming makes signals actionable in real-time
- Independent of: schema bloat (arch: reduce schema bloat at phase boundaries — context-specific view models #106), context pruning (arch: context pruning — pass domain-specific recon slices to hunters #107), reachability (#TBD)