(feat) Add MemoryCoordinator and IndexRegistry for serverless scaling#8
Merged
(feat) Add MemoryCoordinator and IndexRegistry for serverless scaling#8
Conversation
nknize
force-pushed
the
memoryCoordinator
branch
from
3a65747 to
b484d3f
Compare
Introduces two new components for serverless deployments: MemoryCoordinator: - Adaptive memory budget split between cache (40%) and mmap (60%) - Automatic rebalancing based on pressure detection - Workload hints (BulkIngestion, QueryHeavy, MemoryConstrained) - Environment variable configuration (XTREE_MEMORY_BUDGET) IndexRegistry: - Lazy index loading - indexes registered as lightweight metadata - On-demand loading via get_or_load<Record>() - LRU-based cold index eviction under memory pressure - Manifest integration for serverless cold start - register_from_data_dir() for one-line initialization Together these enable scaling to 1000+ indexes in memory-constrained serverless environments (Lambda, Cloud Run) by loading only active indexes and automatically unloading cold ones. 48 tests added, all passing. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
nknize
force-pushed
the
memoryCoordinator
branch
from
b484d3f to
b5d330d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces two new components for serverless deployments:
MemoryCoordinator:
IndexRegistry:
Together these enable scaling to 1000+ indexes in memory-constrained serverless environments (Lambda, Cloud Run) by loading only active indexes and automatically unloading cold ones.
48 tests added, all passing.