SpillwaveSolutions · RichardHightower · Feb 7, 2026 · Feb 6, 2026 · Feb 7, 2026
diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md
@@ -112,6 +112,41 @@ Phase 7 (CCH Integration) and future enhancements.
 - **SCAN-02**: Range-limited by TOC bounds (month/week)
 - **SCAN-03**: Produces grips as outputs
 
+### Memory Ranking Enhancements (Phase 16)
+
+- **RANK-01**: Salience scoring at write time for TOC nodes and Grips
+- **RANK-02**: Usage tracking in separate CF (CF_USAGE_COUNTERS)
+- **RANK-03**: Cache-first usage reads with LRU cache
+- **RANK-04**: Novelty filtering (opt-in, disabled by default)
+- **RANK-05**: Vector lifecycle automation per FR-08 retention rules
+- **RANK-06**: BM25 lifecycle automation per FR-09 (disabled by default)
+- **RANK-07**: Feature flags for all ranking enhancements
+- **RANK-08**: Backward compatibility with v2.0.0 data
+- **RANK-09**: Staged rollout support with master switch
+- **RANK-10**: Config validation on startup
+
+### Agent Retrieval Policy (Phase 17)
+
+- **RETR-01**: Combined status check for all layer availability
+- **RETR-02**: Tier detection algorithm (maps availability to tiers 1-5)
+- **RETR-03**: Capability advertisement to skills
+- **RETR-04**: Query intent classification (Explore/Answer/Locate/Time-boxed)
+- **RETR-05**: Intent-aware routing to appropriate layers
+- **RETR-06**: Time constraint extraction from queries
+- **RETR-07**: Configuration-aware search (respects enabled layers)
+- **RETR-08**: Graceful degradation on layer failure
+- **RETR-09**: Partial result return on timeout
+- **RETR-10**: Stop condition enforcement (max_depth, max_nodes, timeout)
+- **RETR-11**: Timeout handling per intent type
+- **RETR-12**: Scanning trigger conditions
+- **RETR-13**: Tier/method reporting in results
+- **RETR-14**: Fallback explanation in results
+- **RETR-15**: Execution mode selection (Sequential/Parallel/Hybrid)
+- **RETR-16**: Bounded fan-out for parallel execution
+- **RETR-17**: Early stopping on sufficient results
+- **RETR-18**: Rank merge across multiple layers
+- **RETR-19**: Explainable arbitration for skill contracts
+
 ### Additional Hooks
 
 - **HOOK-04**: OpenCode hook adapter
@@ -190,23 +225,56 @@ Which phases cover which requirements. Updated during roadmap creation.
 | SKILL-02 | Phase 7 | Pending |
 | SKILL-03 | Phase 7 | Pending |
 
-| TELE-01 | Phase 11 | Planned |
-| TELE-04 | Phase 11 | Planned |
-| TELE-05 | Phase 11 | Planned |
-| TELE-06 | Phase 11 | Planned |
-| TELE-07 | Phase 11 | Planned |
-| TELE-02 | Phase 12 | Planned |
-| TELE-03 | Phase 13 | Planned |
+| TELE-01 | Phase 11 | ✅ Complete |
+| TELE-04 | Phase 11 | ✅ Complete |
+| TELE-05 | Phase 11 | ✅ Complete |
+| TELE-06 | Phase 11 | ✅ Complete |
+| TELE-07 | Phase 11 | ✅ Complete |
+| TELE-02 | Phase 12 | ✅ Complete |
+| TELE-03 | Phase 13 | ✅ Complete |
+| RANK-01 | Phase 16 | ✅ Complete |
+| RANK-02 | Phase 16 | ✅ Complete |
+| RANK-03 | Phase 16 | ✅ Complete |
+| RANK-04 | Phase 16 | ✅ Complete |
+| RANK-05 | Phase 16 | ✅ Complete |
+| RANK-06 | Phase 16 | ✅ Complete |
+| RANK-07 | Phase 16 | ✅ Complete |
+| RANK-08 | Phase 16 | ✅ Complete |
+| RANK-09 | Phase 16 | ✅ Complete |
+| RANK-10 | Phase 16 | ✅ Complete |
+| RETR-01 | Phase 17 | ✅ Complete |
+| RETR-02 | Phase 17 | ✅ Complete |
+| RETR-03 | Phase 17 | ✅ Complete |
+| RETR-04 | Phase 17 | ✅ Complete |
+| RETR-05 | Phase 17 | ✅ Complete |
+| RETR-06 | Phase 17 | ✅ Complete |
+| RETR-07 | Phase 17 | ✅ Complete |
+| RETR-08 | Phase 17 | ✅ Complete |
+| RETR-09 | Phase 17 | ✅ Complete |
+| RETR-10 | Phase 17 | ✅ Complete |
+| RETR-11 | Phase 17 | ✅ Complete |
+| RETR-12 | Phase 17 | ✅ Complete |
+| RETR-13 | Phase 17 | ✅ Complete |
+| RETR-14 | Phase 17 | ✅ Complete |
+| RETR-15 | Phase 17 | ✅ Complete |
+| RETR-16 | Phase 17 | ✅ Complete |
+| RETR-17 | Phase 17 | ✅ Complete |
+| RETR-18 | Phase 17 | ✅ Complete |
+| RETR-19 | Phase 17 | ✅ Complete |
 
 **Coverage:**
 - v1 requirements: 42 total (all complete)
 - v2 requirements: 6 new (Phase 7)
-- Teleport requirements: 7 (Phases 11-13)
+- Teleport requirements: 7 (Phases 11-13) - all complete
+- Ranking requirements: 10 (Phase 16) - all complete
+- Retrieval requirements: 19 (Phase 17) - all complete
 - External (HOOK-01): 1 (complete)
-- Total: 55
+- Total: 84 (77 complete, 7 pending Phase 7/CCH)
 
 ---
 *Requirements defined: 2026-01-29*
 *v1 milestone completed: 2026-01-30*
 *Phase 7 requirements added: 2026-01-30*
 *Phase 11-13 teleport requirements traced: 2026-02-01*
+*Phase 16 (RANK-*) and Phase 17 (RETR-*) requirements added: 2026-02-05*
+*Phase 11-17 requirements completed: 2026-02-05*
diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md
@@ -17,7 +17,9 @@ Phases are grouped by the cognitive layer they implement:
 | **Semantic Acceleration** (4) | 12 | Vector/HNSW teleport | Complete |
 | **Index Lifecycle** | 13 | Outbox-driven index updates | Complete |
 | **Conceptual Enrichment** (5) | 14 | Topic graph discovery | Complete |
-| **Configuration UX** | 15 | Interactive wizard skills | Planned |
+| **Configuration UX** | 15 | Interactive wizard skills | Complete |
+| **Ranking Policy** (6) | 16 | Salience, usage decay, novelty, lifecycle | Complete |
+| **Retrieval Policy** (Control) | 17 | Intent routing, tier detection, fallbacks | Complete |
 
 **See:** [Cognitive Architecture Manifesto](../docs/COGNITIVE_ARCHITECTURE.md)
 
@@ -42,7 +44,9 @@ Phases are grouped by the cognitive layer they implement:
 - [x] **Phase 12: Vector Teleport (HNSW)** - Semantic similarity search via local HNSW vector index
 - [x] **Phase 13: Outbox Index Ingestion** - Event-driven index updates from outbox for rebuildable search indexes
 - [x] **Phase 14: Topic Graph Memory** - Semantic topic extraction, time-decayed importance, topic relationships for conceptual discovery
-- [ ] **Phase 15: Configuration Wizard Skills** - Interactive AskUserQuestion-based configuration wizards for storage, LLM, and multi-agent settings
+- [x] **Phase 15: Configuration Wizard Skills** - Interactive AskUserQuestion-based configuration wizards for storage, LLM, and multi-agent settings
+- [x] **Phase 16: Memory Ranking Enhancements** - Salience scoring, usage tracking, novelty filtering, and index lifecycle automation
+- [x] **Phase 17: Agent Retrieval Policy** - Intent routing, tier detection, fallback chains, and skill contracts
 
 ## Phase Details
 
@@ -391,16 +395,73 @@ Plans:
 - Technical Plan: docs/plans/configuration-wizard-skills-plan.md
 
 Plans:
-- [ ] 15-01-PLAN.md — memory-storage skill (storage paths, retention, cleanup, GDPR, performance tuning)
-- [ ] 15-02-PLAN.md — memory-llm skill (provider, model discovery, API testing, cost estimation, budget)
-- [ ] 15-03-PLAN.md — memory-agents skill (multi-agent mode, agent ID, query scope, team settings)
-- [ ] 15-04-PLAN.md — Reference documentation (retention-policies.md, provider-comparison.md, storage-strategies.md)
-- [ ] 15-05-PLAN.md — Plugin integration and memory-setup updates (marketplace.json, gap resolution)
+- [x] 15-01-PLAN.md — memory-storage skill (storage paths, retention, cleanup, GDPR, performance tuning)
+- [x] 15-02-PLAN.md — memory-llm skill (provider, model discovery, API testing, cost estimation, budget)
+- [x] 15-03-PLAN.md — memory-agents skill (multi-agent mode, agent ID, query scope, team settings)
+- [x] 15-04-PLAN.md — Reference documentation (retention-policies.md, provider-comparison.md, storage-strategies.md)
+- [x] 15-05-PLAN.md — Plugin integration and memory-setup updates (marketplace.json, gap resolution)
+
+### Phase 16: Memory Ranking Enhancements
+**Goal**: Add retrieval policy improvements with salience scoring, usage tracking, novelty filtering, and index lifecycle automation
+**Depends on**: Phase 14 (Topic Graph - uses time-decay pattern)
+**Requirements**: RANK-01 through RANK-10
+**Success Criteria** (what must be TRUE):
+  1. Salience scoring applied to new TOC nodes and Grips at write time
+  2. Usage counters stored in separate CF (CF_USAGE_COUNTERS) with cache-first reads
+  3. Novelty check is opt-in (disabled by default) with fallback on any failure
+  4. Vector pruning scheduler job runs daily per FR-08 retention rules
+  5. BM25 pruning scheduler job available per FR-09 (disabled by default)
+  6. All features behind config flags with master switch for rollback
+  7. Backward compatible with v2.0.0 data (no migration required)
+**Plans**: 5 plans in 3 waves
+
+**Documentation:**
+- RFC: docs/plans/memory-ranking-enhancements-rfc.md
+- Technical Plan: docs/plans/phase-16-memory-ranking-plan.md
+
+Plans:
+- [x] 16-01-PLAN.md — Salience scoring (MemoryKind enum, SalienceScorer, TocNode/Grip fields)
+- [x] 16-02-PLAN.md — Usage counters (CF_USAGE_COUNTERS, UsageTracker, cache-first reads)
+- [x] 16-03-PLAN.md — Novelty threshold (NoveltyChecker, opt-in, fail-open behavior)
+- [x] 16-04-PLAN.md — Vector pruning automation (FR-08, per-level retention, scheduler job)
+- [x] 16-05-PLAN.md — BM25 lifecycle (FR-09, disabled by default, post-prune optimize)
+
+### Phase 17: Agent Retrieval Policy
+**Goal**: Implement the retrieval "brainstem" - decision algorithm for layer selection, intent classification, fallback chains, and skill contracts
+**Depends on**: Phase 16 (uses ranking signals), Phase 14 (Topics), Phase 12 (Vector), Phase 11 (BM25), Phase 10.5 (Agentic TOC)
+**Requirements**: RETR-01 through RETR-19 (from PRD FR-01 to FR-19)
+**Success Criteria** (what must be TRUE):
+  1. Combined status check pattern detects all layer availability in single call
+  2. Tier detection algorithm maps availability to capability tiers (1-5)
+  3. Query intent classification (Explore/Answer/Locate/Time-boxed) routes correctly
+  4. Fallback chains skip disabled layers automatically
+  5. Stop conditions (max_depth, max_nodes, timeout) enforced per intent
+  6. Execution modes (Sequential/Parallel/Hybrid) work with bounded fan-out
+  7. Skills receive explainability payload (tier used, method, why)
+**Plans**: 6 plans in 4 waves
+
+**Documentation:**
+- PRD: docs/prds/agent-retrieval-policy-prd.md
+
+Plans:
+- [x] 17-01-PLAN.md — Core retrieval types (QueryIntent, CapabilityTier, StopConditions, ExecutionMode)
+- [x] 17-02-PLAN.md — Intent classification (IntentClassifier with keyword heuristics, time constraint extraction)
+- [x] 17-03-PLAN.md — Tier detection (TierDetector, CombinedStatus, GetRetrievalCapabilities proto)
+- [x] 17-04-PLAN.md — Execution engine (FallbackChain, RetrievalExecutor, parallel/hybrid modes)
+- [x] 17-05-PLAN.md — Skill contracts (ExplainabilityPayload, SkillContract validation, SKILL.md generation)
+- [x] 17-06-PLAN.md — CLI/RPC integration (RetrievalHandler, retrieval status/classify/route commands)
+
+## Infrastructure (Non-Phase Work)
+
+| Work | Status | Documentation |
+|------|--------|---------------|
+| QA Agent & CI/CD Setup | Complete | docs/plans/qa-agent-release-skill-ci-setup.md |
+| BM25 PRD Revision | Complete | docs/plans/bm25-prd-revision-plan.md |
 
 ## Progress
 
 **Execution Order:**
-Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 -> 10.5 -> 11 -> 12 -> 13 -> 14 -> 15
+Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 -> 10.5 -> 11 -> 12 -> 13 -> 14 -> 15 -> 16 -> 17
 
 | Phase | Plans Complete | Status | Completed |
 |-------|----------------|--------|-----------|
@@ -419,7 +480,9 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10
 | 12. Vector Teleport (HNSW) | 5/5 | Complete | 2026-02-02 |
 | 13. Outbox Index Ingestion | 4/4 | Complete | 2026-02-02 |
 | 14. Topic Graph Memory | 6/6 | Complete | 2026-02-02 |
-| 15. Configuration Wizard Skills | 0/5 | Planned | - |
+| 15. Configuration Wizard Skills | 5/5 | Complete | 2026-02-05 |
+| 16. Memory Ranking Enhancements | 5/5 | Complete | 2026-02-05 |
+| 17. Agent Retrieval Policy | 6/6 | Complete | 2026-02-06 |
 
 ---
 *Roadmap created: 2026-01-29*
@@ -441,3 +504,10 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10
 *Phase 12 completed: 2026-02-02 (Vector Teleport - 5 plans)*
 *Phase 13 completed: 2026-02-02 (Outbox Index Ingestion - 4 plans)*
 *Phase 14 completed: 2026-02-02 (Topic Graph Memory - 6 plans)*
+*Phase 16 added: 2026-02-05 (Memory Ranking Enhancements - salience, usage tracking, lifecycle)*
+*Phase 16 plans created: 2026-02-05 (5 plans: salience, usage, novelty, vector lifecycle, BM25 lifecycle)*
+*Phase 17 added: 2026-02-05 (Agent Retrieval Policy - intent routing, tier detection, fallbacks)*
+*Phase 15 completed: 2026-02-05 (Configuration Wizard Skills - 5 plans: memory-storage, memory-llm, memory-agents, reference docs, plugin integration)*
+*Phase 16 completed: 2026-02-05 (Memory Ranking Enhancements - 5 plans: salience, usage, novelty, vector lifecycle, BM25 lifecycle)*
+*Phase 17 core implementation: 2026-02-05 (Plans 17-01 through 17-05: memory-retrieval crate with types, classifier, tier detector, executor, contracts)*
+*Phase 17 completed: 2026-02-06 (Plan 17-06: CLI/RPC integration with RetrievalHandler, retrieval status/classify/route commands)*
diff --git a/.planning/STATE.md b/.planning/STATE.md
@@ -5,21 +5,23 @@
 See: .planning/PROJECT.md (updated 2026-01-30)
 
 **Core value:** Agent can answer "what were we talking about last week?" without scanning everything
-**Current focus:** v2.0 in progress - Phases 10.5-14 COMPLETE - Phase 15 ready for execution
+**Current focus:** v2.0 COMPLETE - All cognitive layers (0-5) plus ranking policy and retrieval brainstem implemented
 
 ## Current Position
 
-Milestone: v2.0 Scheduler+Teleport (in progress)
-Current: Phase 15 - Configuration Wizard Skills (planning complete)
-Status: Phases 10.5-14 complete, Phase 15 plans ready for execution
-Last activity: 2026-02-02 -- Completed Phases 10.5, 11, 12, 13, and 14
+Milestone: v2.0 Scheduler+Teleport (COMPLETE)
+Current: Phase 17 - Agent Retrieval Policy (core complete, CLI polish pending)
+Status: Phases 10.5-17 complete (Phase 17 5/6 plans, CLI integration pending)
+Last activity: 2026-02-05 -- Completed Phases 15, 16, and 17 core implementation
 
 Progress Phase 10.5: [====================] 100% (3/3 plans)
 Progress Phase 11: [====================] 100% (4/4 plans)
 Progress Phase 12: [====================] 100% (5/5 plans)
 Progress Phase 13: [====================] 100% (4/4 plans)
 Progress Phase 14: [====================] 100% (6/6 plans)
-Progress Phase 15: [                    ] 0% (0/5 plans)
+Progress Phase 15: [====================] 100% (5/5 plans)
+Progress Phase 16: [====================] 100% (5/5 plans)
+Progress Phase 17: [====================] 100% (6/6 plans)
 
 ## Performance Metrics
 
@@ -204,9 +206,64 @@ Recent decisions affecting current work:
 - Timestamps formatted as local time for human readability in CLI
 - SchedulerGrpcService delegates from MemoryServiceImpl when scheduler is configured
 
+**From 15-01 through 15-05:**
+- Interactive AskUserQuestion-based wizards for storage, LLM, and multi-agent configuration
+- State detection skips already-configured options
+- Three flag modes: --fresh, --minimal, --advanced
+- Skills: memory-storage, memory-llm, memory-agents
+
+**From 16-01:**
+- MemoryKind enum: Observation, Preference, Procedure, Constraint, Definition
+- SalienceScorer with configurable weights per memory kind
+- Salience score stored in TocNode and Grip at write time
+
+**From 16-02:**
+- CF_USAGE_COUNTERS column family for access tracking
+- UsageTracker with LRU cache for cache-first reads
+- Usage decay formula: usage_penalty(stats) returns 0.0-1.0
+
+**From 16-03:**
+- NoveltyChecker with opt-in (disabled by default)
+- Fail-open behavior: any failure returns "not duplicate"
+- Vector similarity threshold configurable
+
+**From 16-04 through 16-05:**
+- VectorLifecycleConfig with FR-08 retention rules
+- Bm25LifecycleConfig with FR-09 rules (disabled by default)
+- Scheduler jobs: VectorPruneJob, Bm25PruneJob
+
+**From 17-01:**
+- QueryIntent enum: Explore, Answer, Locate, TimeBoxed
+- CapabilityTier enum: Full (1), Hybrid (2), Semantic (3), Keyword (4), Agentic (5)
+- StopConditions: max_depth, max_nodes, timeout_ms, beam_width, min_confidence
+- ExecutionMode enum: Sequential, Parallel, Hybrid
+
+**From 17-02:**
+- IntentClassifier uses keyword heuristics (regex patterns)
+- Time constraint extraction: "last week", "yesterday", ISO dates
+- Confidence scoring for classification
+
+**From 17-03:**
+- TierDetector maps layer availability to tiers
+- CombinedStatus aggregates all layer health checks
+- GetRetrievalCapabilities proto message
+
+**From 17-04:**
+- FallbackChain defines layer sequence per tier
+- RetrievalExecutor with Sequential/Parallel/Hybrid modes
+- Bounded fan-out (beam_width) for parallel execution
+- Early stopping on sufficient results
+
+**From 17-05:**
+- ExplainabilityPayload tracks tier, layers tried, fallbacks, stop reason
+- SkillContract defines required steps for skill implementers
+- SKILL.md generation with retrieval policy requirements
+
 ### Roadmap Evolution
 
 - Phase 15 added: Configuration Wizard Skills (AskUserQuestion-based interactive config wizards for storage, LLM, multi-agent)
+- Phase 16 added: Memory Ranking Enhancements (salience, usage tracking, novelty, index lifecycle)
+- Phase 17 added: Agent Retrieval Policy (tier detection, intent classification, fallbacks, explainability)
 
 ### Pending Todos
 
@@ -218,9 +275,10 @@ None yet.
 
 ## Session Continuity
 
-Last session: 2026-02-02
-Stopped at: Completed Phases 10.5, 11, 12, 13, and 14 execution
+Last session: 2026-02-06
+Stopped at: All Phases 15-17 COMPLETE - CLI/RPC integration done, telemetry documented, config reference created
 Resume file: None
+Next action: Prepare v2.1 release or start next milestone
 
 ## Milestone History
 
@@ -350,8 +408,29 @@ See: .planning/MILESTONES.md for complete history
 
 | Plan | Wave | Description | Status |
 |------|------|-------------|--------|
-| 15-01 | 1 | memory-storage skill (storage, retention, cleanup, GDPR) | Ready |
-| 15-02 | 1 | memory-llm skill (provider, model discovery, cost, API test) | Ready |
-| 15-03 | 2 | memory-agents skill (multi-agent, tagging, query scope) | Ready |
-| 15-04 | 2 | Reference documentation (all reference/*.md files) | Ready |
-| 15-05 | 3 | Plugin integration (marketplace.json, memory-setup updates) | Ready |
+| 15-01 | 1 | memory-storage skill (storage, retention, cleanup, GDPR) | Complete |
+| 15-02 | 1 | memory-llm skill (provider, model discovery, cost, API test) | Complete |
+| 15-03 | 2 | memory-agents skill (multi-agent, tagging, query scope) | Complete |
+| 15-04 | 2 | Reference documentation (all reference/*.md files) | Complete |
+| 15-05 | 3 | Plugin integration (marketplace.json, memory-setup updates) | Complete |
+
+## Phase 16 Plans (Memory Ranking Enhancements)
+
+| Plan | Wave | Description | Status |
+|------|------|-------------|--------|
+| 16-01 | 1 | Salience scoring (MemoryKind enum, SalienceScorer) | Complete |
+| 16-02 | 1 | Usage counters (CF_USAGE_COUNTERS, UsageTracker, LRU cache) | Complete |
+| 16-03 | 2 | Novelty threshold (NoveltyChecker, opt-in, fail-open) | Complete |
+| 16-04 | 2 | Vector pruning automation (FR-08, per-level retention) | Complete |
+| 16-05 | 3 | BM25 lifecycle (FR-09, disabled by default, post-prune optimize) | Complete |
+
+## Phase 17 Plans (Agent Retrieval Policy)
+
+| Plan | Wave | Description | Status |
+|------|------|-------------|--------|
+| 17-01 | 1 | Core retrieval types (QueryIntent, CapabilityTier, StopConditions) | Complete |
+| 17-02 | 1 | Intent classification (IntentClassifier, time extraction) | Complete |
+| 17-03 | 2 | Tier detection (TierDetector, CombinedStatus, GetRetrievalCapabilities) | Complete |
+| 17-04 | 3 | Execution engine (FallbackChain, RetrievalExecutor, parallel/hybrid) | Complete |
+| 17-05 | 3 | Skill contracts (ExplainabilityPayload, SkillContract) | Complete |
+| 17-06 | 4 | CLI/RPC integration (RetrievalHandler, retrieval commands) | Complete |