feat: add configurable local data directory with stale lock recovery#4
Merged
frperezr merged 1 commit intokraklabs:mainfrom Feb 14, 2026
Merged
Conversation
Allow users to customize where CIE stores indexed data via
`indexing.local_data_dir` config or `CIE_DATA_DIR` env var, instead of
hardcoding `~/.cie/data/`. Relative paths resolve from the project config
directory. Includes legacy data dir detection with migration warning,
stale RocksDB LOCK file recovery, graceful shutdown improvements
(signal.Stop + user message), and build fixes (GOPATH/bin, macOS
codesign, CGO LDFLAGS with ${SRCDIR}).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Member
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. Tests pass across all packages. |
frperezr
added a commit
that referenced
this pull request
Feb 14, 2026
- Remove unused getCIEDir() function that golangci-lint flagged
- Fix Linux build: copy platform-specific libcozo_c.a to repo lib/
instead of system paths, so CGO LDFLAGS -L${SRCDIR}/../../lib
finds the correct library (not the vendored macOS one)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add support for a configurable local data directory, replacing the hardcoded
~/.cie/data/path. Users can now control where CIE stores indexed data via theindexing.local_data_dirconfig field or theCIE_DATA_DIRenvironment variable. Relative paths are resolved from the directory containing.cie/project.yaml.Additionally includes stale RocksDB LOCK file recovery (auto-removes orphaned locks from crashed processes), graceful shutdown improvements, and build system fixes.
Related Issue
N/A
Type of Change
Testing
Tests added
cmd/cie/paths_test.go— unit tests forprojectDataDir,dataRootFromConfig, path resolution (absolute, relative, default fallback)pkg/storage/embedded_test.go—TestIsStaleLock_NoLockFile,TestIsStaleLock_StaleLock,TestIsStaleLock_ActiveLockChecklist
make fmtandmake lintKey Changes
Configurable data directory
indexing.local_data_dir: "/mnt/cie-data"/<project_id>appendedindexing.local_data_dir: ".cie-data"CIE_DATA_DIR=/mnt/cie-data~/.cie/data/<project_id>/(unchanged behavior)Legacy data migration warning
When
local_data_diris configured and data exists at the old default path (~/.cie/data/<project_id>/) but not at the new path, CIE prints a warning guiding the user to re-index.Stale RocksDB LOCK recovery (
pkg/storage/embedded.go)If CozoDB fails to open and a
LOCKfile exists, CIE checks whether any process actually holds theflock. If not (crashed/killed process), it removes the stale lock and retries. This eliminates the need for users to manually delete lock files after crashes.Graceful shutdown improvements
signal.Stop(sigChan)so a second Ctrl+C force-quits immediately"Shutting down... Press Ctrl+C again to force quit."index,mcp, andservecommandsBuild fixes (
Makefile,pkg/cozodb/cozodb.go)INSTALL_DIRuses$(GOPATH)/bininstead of$(HOME)/go/bincodesign --force --sign -)-L${SRCDIR}/../../libsogo install ./cmd/ciefinds the vendored static libraryRefactoring
cmd/cie/paths.go(replaces scatteredfilepath.Join(homeDir, ".cie", "data", ...)calls)checkLocalDatalogic intoLocalPipeline.FunctionCount()— queries through the already-open DB instead of opening a second connectionstorage.EmbeddedConfig.DataDiris now passed explicitly instead of being computed internallyAdditional Notes
local_data_diris not set, behavior is identical to beforeEmbeddedConfig.DataDirfield replaces the internal~/.cie/data/<project>computation instorage.NewEmbeddedBackend, giving callers full control over the path