Skip to content

Conversation

@RichardHightower
Copy link
Contributor

Summary

  • Lifecycle Pruning: BM25 and Vector indexes now have scheduled pruning jobs that actually delete old data
  • Daemon Wiring: register_prune_jobs() creates jobs with real callbacks at startup
  • Telemetry: Registry tracks job metadata; GetRankingStatusResponse includes lifecycle stats
  • Toolchain Fix: Added env.sh + Taskfile.yml for macOS C++ header issues
  • New Crate: memory-retrieval with classifier and executor for agentic search
  • Planning: Phase 16/17 docs for ranking enhancements and retrieval policies

Key Changes

Component Change
memory-scheduler BM25/Vector prune jobs with callbacks
memory-daemon register_prune_jobs() wiring
memory-search prune(), prune_and_commit() methods
memory-vector prune_level() method
proto/memory.proto Lifecycle fields in GetRankingStatusResponse
Taskfile.yml Task runner for build/test/clippy/check

Test Plan

  • All 735 tests pass
  • Clippy clean
  • Fixed flaky scheduler timeout test

🤖 Generated with Claude Code

RichardHightower and others added 2 commits February 6, 2026 17:52
- Add BM25 prune jobs with SearchIndexer::prune_and_commit callback
- Add Vector prune jobs with VectorIndexPipeline::prune_level callback
- Wire prune jobs at daemon startup via register_prune_jobs()
- Add registry metadata tracking for job telemetry
- Extend proto with GetRankingStatusResponse lifecycle fields
- Add env.sh and Taskfile.yml for macOS C++ toolchain workaround
- Fix flaky scheduler timeout test (use 5s cron interval)
- Add memory-retrieval crate with classifier and executor
- Add phase 16/17 planning docs for ranking enhancements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix unused `merged` field in UsageUpdate (prefix with _)
- Replace manual Default impl with derive for Bm25PruneJobConfig
- Remove unused imports in retrieval.rs
- Prefix unused variables with underscore
- Handle all match arms explicitly in layer_status_from_proto

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@RichardHightower RichardHightower merged commit 599e752 into main Feb 7, 2026
8 checks passed
@RichardHightower RichardHightower deleted the feature/bm25-index-lifecycle-prd branch February 7, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant