diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md deleted file mode 100644 index 02ba138..0000000 --- a/.claude/CLAUDE.md +++ /dev/null @@ -1,99 +0,0 @@ -# Squirrel Project - -Local-first memory system for AI coding tools. - -## AI-First Development - -**This project is 98% AI-coded.** You are the primary developer. - -All documentation, specs, and structures are designed for AI comprehension. Use declarative thinking: specs define WHAT, you implement HOW. - -| Principle | Meaning | -|-----------|---------| -| Specs are source of truth | Never implement undefined behavior | -| Declarative over imperative | Define outcomes, not steps | -| Tables over prose | Structured data > paragraphs | -| Stable IDs everywhere | SCHEMA-001, IPC-001, ADR-001 | -| Update specs first | Specs change before code changes | - -## Spec-Driven Development - -**Specs are the source of truth. Code is compiled output.** - -| Spec File | Purpose | -|-----------|---------| -| specs/CONSTITUTION.md | Project governance, core principles | -| specs/ARCHITECTURE.md | System boundaries, data flow | -| specs/SCHEMAS.md | Database schemas (SCHEMA-*) | -| specs/INTERFACES.md | IPC, MCP, CLI contracts (IPC-*, MCP-*, CLI-*) | -| specs/KEYS.md | Declarative key registry (KEY-*) | -| specs/PROMPTS.md | LLM prompts with model tiers (PROMPT-*) | -| specs/DECISIONS.md | Architecture decision records (ADR-*) | - -**Rules:** -1. Read specs before implementing -2. Never implement behavior not defined in specs -3. Update specs before or with code, never after -4. Reference spec IDs in commits (e.g., "implements SCHEMA-001") - -## Project Rules - -See `project-rules/*.mdc` for context-specific rules: -- `general.mdc` - Overall development rules -- `rust-daemon.mdc` - Rust daemon boundaries (ARCH-001) -- `python-agent.mdc` - Python agent boundaries (ARCH-002) -- `specs.mdc` - Specification maintenance -- `testing.mdc` - Testing requirements (DR4) - -## Architecture - -``` -Rust Daemon (I/O, storage, MCP) <--IPC--> Python Agent (LLM operations) -``` - -| Component | Responsibility | Never Does | -|-----------|----------------|------------| -| Rust Daemon | Log watching, MCP server, CLI, SQLite | LLM calls | -| Python Agent | Memory extraction, context composition | File watching | - -## Development Environment - -Uses Nix via devenv (ADR-006). Single command: - -```bash -devenv shell -``` - -Available commands: -- `test-all` - Run all tests -- `dev-daemon` - Start daemon in dev mode -- `fmt` - Format all code -- `lint` - Lint all code - -## Team Standards - -### Communication -- No unnecessary emojis -- Documentation written for AI comprehension -- English only in code, comments, commits -- Brief, direct language -- Today's date: 2025 Dec 9 - -### Git Workflow - -Branch: `yourname/type-description` -- `feat`, `fix`, `refactor`, `docs`, `test`, `chore` - -Commit: `type(scope): brief description` -- Reference spec IDs when applicable - -### Code Quality -- Write tests for new features (DR4) -- Keep files under 200 lines -- Only change what's necessary (DR5) -- No drive-by refactoring - -### Security -- Never commit secrets -- Validate user input -- Review AI-generated code diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md new file mode 120000 index 0000000..be77ac8 --- /dev/null +++ b/.claude/CLAUDE.md @@ -0,0 +1 @@ +../AGENTS.md \ No newline at end of file diff --git a/project-rules/general.mdc b/.cursor/rules/general.mdc similarity index 100% rename from project-rules/general.mdc rename to .cursor/rules/general.mdc diff --git a/project-rules/python-agent.mdc b/.cursor/rules/python-agent.mdc similarity index 100% rename from project-rules/python-agent.mdc rename to .cursor/rules/python-agent.mdc diff --git a/project-rules/rust-daemon.mdc b/.cursor/rules/rust-daemon.mdc similarity index 100% rename from project-rules/rust-daemon.mdc rename to .cursor/rules/rust-daemon.mdc diff --git a/project-rules/specs.mdc b/.cursor/rules/specs.mdc similarity index 100% rename from project-rules/specs.mdc rename to .cursor/rules/specs.mdc diff --git a/project-rules/testing.mdc b/.cursor/rules/testing.mdc similarity index 100% rename from project-rules/testing.mdc rename to .cursor/rules/testing.mdc diff --git a/.cursorrules b/.cursorrules deleted file mode 100644 index 6d810b2..0000000 --- a/.cursorrules +++ /dev/null @@ -1,86 +0,0 @@ -# Squirrel Project - -Local-first memory system for AI coding tools. - -## AI-First Development - -**This project is 98% AI-coded.** You are the primary developer. - -All documentation, specs, and structures are designed for AI comprehension. Use declarative thinking: specs define WHAT, you implement HOW. - -| Principle | Meaning | -|-----------|---------| -| Specs are source of truth | Never implement undefined behavior | -| Declarative over imperative | Define outcomes, not steps | -| Tables over prose | Structured data > paragraphs | -| Stable IDs everywhere | SCHEMA-001, IPC-001, ADR-001 | -| Update specs first | Specs change before code changes | - -## Spec-Driven Development - -**Specs are the source of truth. Code is compiled output.** - -| Spec File | Purpose | -|-----------|---------| -| specs/CONSTITUTION.md | Project governance, core principles | -| specs/ARCHITECTURE.md | System boundaries, data flow | -| specs/SCHEMAS.md | Database schemas (SCHEMA-*) | -| specs/INTERFACES.md | IPC, MCP, CLI contracts (IPC-*, MCP-*, CLI-*) | -| specs/KEYS.md | Declarative key registry (KEY-*) | -| specs/PROMPTS.md | LLM prompts with model tiers (PROMPT-*) | -| specs/DECISIONS.md | Architecture decision records (ADR-*) | - -**Rules:** -1. Read specs before implementing -2. Never implement behavior not defined in specs -3. Update specs before or with code, never after -4. Reference spec IDs in commits (e.g., "implements SCHEMA-001") - -## Project Rules - -See `project-rules/*.mdc` for context-specific rules: -- `general.mdc` - Overall development rules -- `rust-daemon.mdc` - Rust daemon boundaries (ARCH-001) -- `python-agent.mdc` - Python agent boundaries (ARCH-002) -- `specs.mdc` - Specification maintenance -- `testing.mdc` - Testing requirements (DR4) - -## Architecture - -``` -Rust Daemon (I/O, storage, MCP) <--IPC--> Python Agent (LLM operations) -``` - -| Component | Responsibility | Never Does | -|-----------|----------------|------------| -| Rust Daemon | Log watching, MCP server, CLI, SQLite | LLM calls | -| Python Agent | Memory extraction, context composition | File watching | - -## Development Environment - -Uses Nix via devenv (ADR-006). Single command: - -```bash -devenv shell -``` - -## Team Standards - -### Communication -- No unnecessary emojis -- English only in code, comments, commits -- Brief, direct language -- Today's date: 2025 Dec 9 - -### Git Workflow - -Branch: `yourname/type-description` - -Commit: `type(scope): brief description` -- Reference spec IDs when applicable - -### Code Quality -- Write tests for new features (DR4) -- Keep files under 200 lines -- Only change what's necessary (DR5) -- No drive-by refactoring diff --git a/.githooks/pre-commit b/.githooks/pre-commit new file mode 100755 index 0000000..cba8ef6 --- /dev/null +++ b/.githooks/pre-commit @@ -0,0 +1,62 @@ +#!/bin/bash +# Pre-commit hook: Reminds AI to check if related docs need updates +# +# This hook does NOT block commits. It only outputs a reminder. +# The AI must decide which docs are related and whether to update them. + +# Get staged files (files in this commit) +STAGED=$(git diff --cached --name-only) + +if [ -z "$STAGED" ]; then + exit 0 +fi + +# Check if code files changed +CODE_CHANGED=$(echo "$STAGED" | grep -E '\.(rs|py|toml|nix)$' || true) + +# Check if spec files changed +SPECS_CHANGED=$(echo "$STAGED" | grep -E '^specs/' || true) + +# Check if doc files changed +DOCS_CHANGED=$(echo "$STAGED" | grep -E '\.(md|mdc)$' || true) + +# Output reminder +echo "" +echo "==========================================" +echo "PRE-COMMIT: Doc Sync Check" +echo "==========================================" +echo "" +echo "Files in this commit:" +echo "$STAGED" | sed 's/^/ - /' +echo "" + +if [ -n "$CODE_CHANGED" ]; then + echo "Code files changed:" + echo "$CODE_CHANGED" | sed 's/^/ - /' + echo "" + echo ">> Check if any specs/*.md need updates" + echo "" +fi + +if [ -n "$SPECS_CHANGED" ]; then + echo "Spec files changed:" + echo "$SPECS_CHANGED" | sed 's/^/ - /' + echo "" + echo ">> Check if related code needs updates" + echo "" +fi + +if [ -n "$DOCS_CHANGED" ] && [ -z "$SPECS_CHANGED" ]; then + echo "Doc files changed:" + echo "$DOCS_CHANGED" | sed 's/^/ - /' + echo "" +fi + +echo "==========================================" +echo "If docs are in sync, proceed with commit." +echo "==========================================" +echo "" + +# Always allow commit (exit 0) +# AI decides whether to abort and fix docs +exit 0 diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..ab1c740 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,35 @@ +--- +name: Bug Report +about: Report a bug in Squirrel +title: '[BUG] ' +labels: bug +assignees: '' +--- + +## Description + + +## Current Behavior + + +## Expected Behavior + + +## Reproduction Steps +1. +2. +3. + +## Environment +- **OS**: +- **Squirrel version**: +- **CLI being used**: + +## Logs + +``` +``` + +## Spec Reference + + diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..97b1315 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,24 @@ +--- +name: Feature Request +about: Suggest a new feature +title: '[FEAT] ' +labels: feat +assignees: '' +--- + +## Problem Statement + + +## Proposed Solution + + +## Alternatives Considered + + +## Spec Impact + +- [ ] New spec needed: +- [ ] Updates to: + +## Additional Context + diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..7522c12 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,43 @@ +# Dependabot configuration +# Implements: SEC-002 (Dependency Security) + +version: 2 +updates: + # Rust dependencies + - package-ecosystem: "cargo" + directory: "/daemon" + schedule: + interval: "weekly" + day: "monday" + open-pull-requests-limit: 5 + labels: + - "dependencies" + - "rust" + commit-message: + prefix: "chore(deps)" + + # Python dependencies + - package-ecosystem: "pip" + directory: "/agent" + schedule: + interval: "weekly" + day: "monday" + open-pull-requests-limit: 5 + labels: + - "dependencies" + - "python" + commit-message: + prefix: "chore(deps)" + + # GitHub Actions + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "weekly" + day: "monday" + open-pull-requests-limit: 3 + labels: + - "dependencies" + - "ci" + commit-message: + prefix: "chore(ci)" diff --git a/.github/markdown-link-check.json b/.github/markdown-link-check.json new file mode 100644 index 0000000..8cb12e9 --- /dev/null +++ b/.github/markdown-link-check.json @@ -0,0 +1,12 @@ +{ + "ignorePatterns": [ + { + "pattern": "^https://sqrl.dev" + }, + { + "pattern": "^#" + } + ], + "replacementPatterns": [], + "aliveStatusCodes": [200, 206, 301, 302, 307, 308] +} diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..a84b754 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,32 @@ +## Summary + + +## Spec References + +- Implements: +- Updates: + +## Type +- [ ] `feat`: New feature +- [ ] `fix`: Bug fix +- [ ] `docs`: Documentation only +- [ ] `refactor`: Code refactoring +- [ ] `test`: Test additions +- [ ] `chore`: Build/tooling changes + +## Checklist +- [ ] Specs updated (if behavior changed) +- [ ] Tests added/updated +- [ ] `fmt` passes locally +- [ ] `lint` passes locally +- [ ] `test-all` passes locally + +## Test Plan + + +--- + diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..027fd64 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,204 @@ +# Squirrel CI Pipeline +# Runs on all PRs and pushes to main +# Implements: DR4 (Test Before Merge) + +name: CI + +on: + push: + branches: [main] + pull_request: + branches: [main] + +env: + CARGO_TERM_COLOR: always + RUST_BACKTRACE: 1 + +jobs: + # ==================== + # Rust Daemon Checks + # ==================== + rust-check: + name: Rust Check + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest, windows-latest] + + steps: + - uses: actions/checkout@v4 + + - name: Install Rust toolchain + uses: dtolnay/rust-action@stable + with: + components: rustfmt, clippy + + - name: Cache cargo registry + uses: actions/cache@v4 + with: + path: | + ~/.cargo/registry + ~/.cargo/git + target + key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} + restore-keys: | + ${{ runner.os }}-cargo- + + - name: Check formatting + run: cargo fmt --all -- --check + working-directory: daemon + + - name: Clippy lints + run: cargo clippy --all-targets --all-features -- -D warnings + working-directory: daemon + + - name: Run tests + run: cargo test --all-features + working-directory: daemon + + - name: Build release + run: cargo build --release + working-directory: daemon + + # ==================== + # Python Agent Checks + # ==================== + python-check: + name: Python Check + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest, windows-latest] + python-version: ["3.11", "3.12"] + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Cache pip packages + uses: actions/cache@v4 + with: + path: ~/.cache/pip + key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('**/requirements*.txt', '**/pyproject.toml') }} + restore-keys: | + ${{ runner.os }}-pip-${{ matrix.python-version }}- + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install ruff pytest pytest-asyncio pytest-cov + pip install -e ".[dev]" + working-directory: agent + + - name: Ruff format check + run: ruff format --check . + working-directory: agent + + - name: Ruff lint + run: ruff check . + working-directory: agent + + - name: Run tests with coverage + run: pytest --cov=squirrel_agent --cov-report=xml --cov-report=term + working-directory: agent + + - name: Upload coverage + uses: codecov/codecov-action@v4 + if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12' + with: + files: agent/coverage.xml + flags: python + fail_ci_if_error: false + + # ==================== + # Security Checks + # ==================== + security: + name: Security Scan + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Install Rust toolchain + uses: dtolnay/rust-action@stable + + - name: Install cargo-audit + run: cargo install cargo-audit + + - name: Rust security audit + run: cargo audit + working-directory: daemon + continue-on-error: true # Advisory only for now + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install pip-audit + run: pip install pip-audit + + - name: Python security audit + run: pip-audit + working-directory: agent + continue-on-error: true # Advisory only for now + + - name: Secret scanning + uses: trufflesecurity/trufflehog@main + with: + extra_args: --only-verified + + # ==================== + # Spec Validation + # ==================== + specs: + name: Spec Validation + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Check spec ID uniqueness + run: | + echo "Checking for duplicate spec IDs..." + # Extract all spec IDs and check for duplicates + grep -rhoE '(SCHEMA|IPC|MCP|CLI|KEY-[PU]|PROMPT|ADR|ARCH|FLOW|DR|PR|COMMIT|REVIEW|ISSUE|SEC|RELEASE|COMM)-[0-9]+' specs/ | sort | uniq -d > duplicates.txt + if [ -s duplicates.txt ]; then + echo "Duplicate spec IDs found:" + cat duplicates.txt + exit 1 + fi + echo "No duplicate spec IDs found" + + - name: Validate markdown links + uses: gaurav-nelson/github-action-markdown-link-check@v1 + with: + folder-path: 'specs/' + config-file: '.github/markdown-link-check.json' + continue-on-error: true + + # ==================== + # All Checks Gate + # ==================== + all-checks: + name: All Checks Passed + runs-on: ubuntu-latest + needs: [rust-check, python-check, security, specs] + if: always() + + steps: + - name: Check all jobs passed + run: | + if [ "${{ needs.rust-check.result }}" != "success" ] || \ + [ "${{ needs.python-check.result }}" != "success" ]; then + echo "Required checks failed" + exit 1 + fi + echo "All required checks passed" diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..ffa726b --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,183 @@ +# Squirrel Release Pipeline +# Triggered by version tags (v*) +# Implements: RELEASE-001, RELEASE-002 + +name: Release + +on: + push: + tags: + - 'v*' + +permissions: + contents: write + +env: + CARGO_TERM_COLOR: always + +jobs: + # ==================== + # Build Binaries + # ==================== + build: + name: Build ${{ matrix.target }} + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + include: + # Linux + - target: x86_64-unknown-linux-gnu + os: ubuntu-latest + archive: tar.gz + - target: aarch64-unknown-linux-gnu + os: ubuntu-latest + archive: tar.gz + + # macOS + - target: x86_64-apple-darwin + os: macos-latest + archive: tar.gz + - target: aarch64-apple-darwin + os: macos-latest + archive: tar.gz + + # Windows + - target: x86_64-pc-windows-msvc + os: windows-latest + archive: zip + + steps: + - uses: actions/checkout@v4 + + - name: Install Rust toolchain + uses: dtolnay/rust-action@stable + with: + targets: ${{ matrix.target }} + + - name: Install cross-compilation tools (Linux ARM) + if: matrix.target == 'aarch64-unknown-linux-gnu' + run: | + sudo apt-get update + sudo apt-get install -y gcc-aarch64-linux-gnu + + - name: Build daemon + run: cargo build --release --target ${{ matrix.target }} + working-directory: daemon + env: + CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc + + - name: Package (Unix) + if: matrix.os != 'windows-latest' + run: | + mkdir -p dist + cp daemon/target/${{ matrix.target }}/release/sqrl dist/ + cp daemon/target/${{ matrix.target }}/release/sqrl-daemon dist/ + cp README.md LICENSE dist/ + cd dist && tar -czvf ../squirrel-${{ matrix.target }}.${{ matrix.archive }} * + + - name: Package (Windows) + if: matrix.os == 'windows-latest' + run: | + mkdir dist + copy daemon\target\${{ matrix.target }}\release\sqrl.exe dist\ + copy daemon\target\${{ matrix.target }}\release\sqrl-daemon.exe dist\ + copy README.md dist\ + copy LICENSE dist\ + Compress-Archive -Path dist\* -DestinationPath squirrel-${{ matrix.target }}.${{ matrix.archive }} + + - name: Upload artifact + uses: actions/upload-artifact@v4 + with: + name: squirrel-${{ matrix.target }} + path: squirrel-${{ matrix.target }}.${{ matrix.archive }} + + # ==================== + # Build Python Wheel + # ==================== + python-wheel: + name: Build Python Wheel + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install build tools + run: pip install build twine + + - name: Build wheel + run: python -m build + working-directory: agent + + - name: Upload artifact + uses: actions/upload-artifact@v4 + with: + name: python-wheel + path: agent/dist/* + + # ==================== + # Create Release + # ==================== + release: + name: Create Release + runs-on: ubuntu-latest + needs: [build, python-wheel] + + steps: + - uses: actions/checkout@v4 + + - name: Download all artifacts + uses: actions/download-artifact@v4 + with: + path: artifacts + + - name: Generate checksums + run: | + cd artifacts + find . -type f \( -name "*.tar.gz" -o -name "*.zip" -o -name "*.whl" \) -exec sha256sum {} \; > SHA256SUMS.txt + cat SHA256SUMS.txt + + - name: Extract version from tag + id: version + run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT + + - name: Create GitHub Release + uses: softprops/action-gh-release@v1 + with: + name: Squirrel v${{ steps.version.outputs.VERSION }} + draft: true + generate_release_notes: true + files: | + artifacts/**/*.tar.gz + artifacts/**/*.zip + artifacts/**/*.whl + artifacts/SHA256SUMS.txt + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + # ==================== + # Publish to PyPI + # ==================== + pypi: + name: Publish to PyPI + runs-on: ubuntu-latest + needs: [release] + environment: pypi + + steps: + - name: Download Python wheel + uses: actions/download-artifact@v4 + with: + name: python-wheel + path: dist + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + password: ${{ secrets.PYPI_API_TOKEN }} + skip-existing: true diff --git a/AGENTS.md b/AGENTS.md index 6d810b2..26228df 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -29,6 +29,7 @@ All documentation, specs, and structures are designed for AI comprehension. Use | specs/KEYS.md | Declarative key registry (KEY-*) | | specs/PROMPTS.md | LLM prompts with model tiers (PROMPT-*) | | specs/DECISIONS.md | Architecture decision records (ADR-*) | +| specs/CONTRIBUTING.md | Collaboration standards (PR-*, COMMIT-*, etc.) | **Rules:** 1. Read specs before implementing @@ -36,9 +37,43 @@ All documentation, specs, and structures are designed for AI comprehension. Use 3. Update specs before or with code, never after 4. Reference spec IDs in commits (e.g., "implements SCHEMA-001") +## AI Workflow + +Follow these phases in order. Never skip phases. Never jump to code without a spec. + +| Phase | Action | Output | +|-------|--------|--------| +| 1. **Specify** | Define WHAT and WHY. No tech stack yet. | `specs/*.md` updated | +| 2. **Clarify** | Ask questions. Mark unclear areas as `[NEEDS CLARIFICATION: reason]` | Ambiguities resolved | +| 3. **Plan** | Define HOW (tech stack, architecture, file structure) | Implementation plan | +| 4. **Tasks** | Break into ordered, independently testable steps | Task list | +| 5. **Implement** | Execute one task at a time. Test after each. | Working code | + +### Before Every Commit (MANDATORY) + +A pre-commit hook (`.githooks/pre-commit`) will show you which files changed. + +**Your job:** Review the list and decide which docs need updates. + +| Files Changed | Check These Docs | +|---------------|------------------| +| `*.rs` (Rust) | `specs/ARCHITECTURE.md`, `specs/INTERFACES.md`, `specs/SCHEMAS.md` | +| `*.py` (Python) | `specs/ARCHITECTURE.md`, `specs/PROMPTS.md` | +| `specs/*.md` | Related code that implements the spec | +| `*.toml`, `*.nix` | `specs/DECISIONS.md` (if config change is significant) | + +**Checklist:** + +| Check | Action if Yes | +|-------|---------------| +| Code changed? | Update related spec | +| Spec changed? | Update related code | +| New key/schema/interface? | Add to registry with ID | +| Unclear requirement? | Mark `[NEEDS CLARIFICATION]`, ask user | + ## Project Rules -See `project-rules/*.mdc` for context-specific rules: +See `.cursor/rules/*.mdc` for context-specific rules: - `general.mdc` - Overall development rules - `rust-daemon.mdc` - Rust daemon boundaries (ARCH-001) - `python-agent.mdc` - Python agent boundaries (ARCH-002) @@ -64,23 +99,47 @@ Uses Nix via devenv (ADR-006). Single command: devenv shell ``` +Available commands: +- `test-all` - Run all tests +- `dev-daemon` - Start daemon in dev mode +- `fmt` - Format all code +- `lint` - Lint all code + ## Team Standards ### Communication - No unnecessary emojis +- Documentation written for AI comprehension - English only in code, comments, commits - Brief, direct language -- Today's date: 2025 Dec 9 +- Today's date: 2025 Dec 10 ### Git Workflow +See `specs/CONTRIBUTING.md` for full details. + Branch: `yourname/type-description` +- `feat`, `fix`, `refactor`, `docs`, `test`, `chore` Commit: `type(scope): brief description` - Reference spec IDs when applicable +- Scopes: daemon, agent, cli, mcp, ipc, specs, ci + +PR Process: +1. Create branch from main +2. Make changes, run `fmt` + `lint` + `test-all` +3. Push and create PR (use template) +4. CI runs automatically +5. Get review, address feedback +6. Squash merge to main ### Code Quality - Write tests for new features (DR4) - Keep files under 200 lines - Only change what's necessary (DR5) - No drive-by refactoring + +### Security +- Never commit secrets +- Validate user input +- Review AI-generated code diff --git a/DEVELOPMENT_PLAN.md b/DEVELOPMENT_PLAN.md deleted file mode 100644 index 0fbb374..0000000 --- a/DEVELOPMENT_PLAN.md +++ /dev/null @@ -1,1097 +0,0 @@ -# Squirrel Development Plan (v1) - -Modular development plan with Rust daemon + Python Agent communicating via Unix socket IPC. - -## Technology Stack - -| Category | Technology | Notes | -|----------|------------|-------| -| **Storage** | SQLite + sqlite-vec | Local-first, vector search | -| **IPC Protocol** | JSON-RPC 2.0 | MCP-compatible, over Unix socket | -| **MCP SDK** | rmcp | Official Rust SDK (modelcontextprotocol/rust-sdk) | -| **CLI Framework** | clap | Rust CLI parsing | -| **Agent Framework** | PydanticAI | Python agent with tools | -| **LLM Client** | LiteLLM | Multi-provider support | -| **Embeddings** | OpenAI text-embedding-3-small | 1536-dim, API-based | -| **Build/Release** | dist (cargo-dist) | Generates Homebrew, MSI, installers | -| **Auto-update** | axoupdater | dist's official updater | -| **Python Packaging** | PyInstaller | Bundled, zero user deps | -| **Logging** | tracing (Rust), structlog (Python) | Structured logging | - -## Architecture Overview - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ RUST DAEMON │ -│ Log Watcher → Events → Episodes → IPC → Python Agent │ -│ SQLite/sqlite-vec storage │ -│ MCP Server (2 tools) │ -│ Thin CLI (passes to agent) │ -└─────────────────────────────────────────────────────────────────┘ - ↕ Unix socket IPC -┌─────────────────────────────────────────────────────────────────┐ -│ PYTHON AGENT │ -│ ┌───────────────────────────────────────────────────────────┐ │ -│ │ Squirrel Agent (single LLM brain) │ │ -│ │ │ │ -│ │ Tools: │ │ -│ │ ├── Memory Tools │ │ -│ │ │ ├── ingest_episode(events) → memories │ │ -│ │ │ ├── search_memories(query) → results │ │ -│ │ │ ├── get_task_context(task) → relevant memories │ │ -│ │ │ └── forget_memory(id) │ │ -│ │ │ │ │ -│ │ ├── Filesystem Tools │ │ -│ │ │ ├── find_cli_configs() → [claude, codex, ...] │ │ -│ │ │ ├── read_file(path) │ │ -│ │ │ ├── write_file(path, content) │ │ -│ │ │ └── scan_project_logs(project_root) → logs │ │ -│ │ │ │ │ -│ │ ├── Config Tools │ │ -│ │ │ ├── get_mcp_config(cli) → config │ │ -│ │ │ ├── set_mcp_config(cli, server, config) │ │ -│ │ │ ├── init_project(path, options) │ │ -│ │ │ └── get/set_user_profile() │ │ -│ │ │ │ │ -│ │ └── DB Tools │ │ -│ │ ├── query_memories(filters) │ │ -│ │ ├── add_memory(memory) │ │ -│ │ ├── update_memory(id, changes) │ │ -│ │ └── get_stats() │ │ -│ └───────────────────────────────────────────────────────────┘ │ -│ │ -│ Entry points (all go through same agent): │ -│ ├── IPC: daemon sends Episode → agent ingests │ -│ ├── IPC: MCP tool call → agent retrieves │ -│ └── IPC: CLI command → agent executes │ -│ │ -│ API Embeddings (text-embedding-3-small, 1536-dim) │ -│ 2-tier LLM: strong (ingest) + fast (compose, CLI, dedup) │ -└─────────────────────────────────────────────────────────────────┘ -``` - -## Core Insight: Episode Segmentation - -Not all sessions are coding tasks with success/failure outcomes. Sessions include architecture discussions, research, brainstorming - these produce valuable memories but don't fit the success/failure model. - -**Episode → Segments → Memories (single LLM call):** - -1. **Segment the episode** by kind (not by task success/failure) -2. **Extract memories** based on segment kind -3. **Only EXECUTION_TASK segments** get SUCCESS/FAILURE classification - -### Segment Kinds - -| Kind | Description | Outcome Field | Memory Output | -|------|-------------|---------------|---------------| -| `EXECUTION_TASK` | Coding, fixing bugs, running commands | `outcome`: SUCCESS / FAILURE / UNCERTAIN | lesson (with outcome), fact | -| `PLANNING_DECISION` | Architecture, design, tech choices | `resolution`: DECIDED / OPEN | fact, lesson (rationale), profile | -| `RESEARCH_LEARNING` | Learning, exploring docs, asking questions | `resolution`: ANSWERED / PARTIAL | fact, lesson | -| `DISCUSSION` | Brainstorming, market research, chat | (none) | profile, lesson (insights) | - -**Key rule:** SUCCESS/FAILURE only allowed on EXECUTION_TASK. Other kinds never output FAILURE. - -### Success/Failure Detection (EXECUTION_TASK only) - -**Success signals (require evidence):** -- AI says "done" / "complete" + User moves to next task → SUCCESS -- Tests pass, build succeeds → SUCCESS -- User says "thanks", "perfect", "works" → SUCCESS - -**Failure signals (require evidence):** -- Same error reappears after attempted fix → FAILURE -- User says "still broken", "that didn't work" → FAILURE - -**No evidence → UNCERTAIN** (conservative default) - -### Episode Ingestion Output Schema - -```json -{ - "episode_summary": "...", - "segments": [ - { - "id": "seg_1", - "kind": "EXECUTION_TASK", - "title": "Fix auth 500 on null user_id", - "event_range": [12, 33], - "outcome": { - "status": "SUCCESS", - "evidence": ["event#31 tests passed", "event#33 user confirmed"] - } - }, - { - "id": "seg_2", - "kind": "PLANNING_DECISION", - "title": "Choose sync conflict strategy", - "event_range": [34, 44], - "resolution": "DECIDED", - "decision": { - "choice": "server-wins", - "rationale": ["shared team DB", "simplicity"] - } - } - ], - "memories": [ - { - "memory_type": "fact", - "scope": "project", - "key": "project.db.engine", - "value": "PostgreSQL", - "text": "Project uses PostgreSQL 15 via Prisma.", - "evidence_source": "neutral", - "source_segments": ["seg_1"], - "confidence": 0.86 - }, - { - "memory_type": "fact", - "scope": "global", - "key": "user.preferred_style", - "value": "async_await", - "text": "User prefers async/await over callbacks.", - "evidence_source": "success", - "source_segments": ["seg_1"], - "confidence": 0.85 - }, - { - "memory_type": "lesson", - "scope": "project", - "outcome": "failure", - "text": "Validate user_id before DB insert to avoid 500s.", - "source_segments": ["seg_1"], - "confidence": 0.9 - }, - { - "memory_type": "fact", - "scope": "project", - "text": "Auth module handles JWT validation in middleware.", - "evidence_source": "neutral", - "source_segments": ["seg_1"], - "confidence": 0.75 - } - ] -} -``` - -**The LLM-decides-everything approach:** -- One LLM call per Episode (4-hour window) -- LLM segments by kind first, then extracts memories per segment -- No rules engine, no heuristics for task detection -- 100% passive - no user prompts or confirmations - -## Development Tracks - -``` -Phase 0: Scaffolding - | - v -+-------+-------+-------+-------+ -| | | | | -v v v v v -Track A Track B Track C Track D Track E -(Rust (Rust (Python (Python (MCP + -Storage) Daemon) Agent) Tools) CLI) - | | | | | - +---+---+ +---+---+ | - | | | - v v v - Week 1-2 Week 2-3 Week 3-4 - | | | - +-------+-------+-----------+ - | - v - Phase X - Hardening -``` - ---- - -## Phase 0 – Scaffolding - -### Rust Module (`agent/`) - -Dependencies: -- `tokio` (async runtime) -- `rusqlite` + `sqlite-vec` (storage) -- `serde`, `serde_json` (serialization, JSON-RPC 2.0) -- `notify` (file watching) -- `clap` (CLI framework) -- `rmcp` (official MCP SDK - modelcontextprotocol/rust-sdk) -- `tracing` (structured logging) -- `uuid`, `chrono` (ID, timestamps) - -Directory structure: -``` -agent/src/ -├── main.rs -├── lib.rs -├── daemon.rs # Lazy start, idle shutdown -├── watcher.rs # Log file watching -├── storage.rs # SQLite + sqlite-vec -├── events.rs # Event/Episode structs -├── config.rs # Config management -├── mcp.rs # MCP server -└── ipc.rs # Unix socket client -``` - -### Python Module (`memory_service/`) - -Dependencies: -- `pydantic-ai` (agent framework) -- `litellm` (multi-provider LLM support) -- `openai` (embeddings API client) -- `pydantic` (schemas) -- `structlog` (structured logging) - -Directory structure: -``` -memory_service/ -├── squirrel_memory/ -│ ├── __init__.py -│ ├── server.py # Unix socket server -│ ├── agent.py # Unified agent -│ ├── tools/ -│ │ ├── __init__.py -│ │ ├── memory.py # ingest, search, get_context, forget -│ │ ├── filesystem.py # find_cli_configs, read/write_file -│ │ ├── config.py # init_project, mcp_config, user_profile -│ │ └── db.py # query, add, update memories -│ ├── embeddings.py # API embeddings (OpenAI, etc.) -│ ├── retrieval.py # Similarity search -│ └── schemas/ -└── tests/ -``` - ---- - -## Track A – Rust: Storage + Config + Events - -### A1. Storage layer (`storage.rs`) - -SQLite + sqlite-vec initialization: -```sql --- memories table (squirrel.db) -CREATE TABLE memories ( - id TEXT PRIMARY KEY, - project_id TEXT, -- NULL for global/user-scope memories - memory_type TEXT NOT NULL, -- lesson | fact | profile - - -- For lessons (task-level patterns/pitfalls) - outcome TEXT, -- success | failure | uncertain (lesson only) - - -- For facts - fact_type TEXT, -- knowledge | process (fact only, optional) - key TEXT, -- declarative key: project.db.engine, user.preferred_style - value TEXT, -- declarative value: PostgreSQL, async_await - evidence_source TEXT, -- success | failure | neutral | manual (fact only) - support_count INTEGER, -- approx episodes that support this fact - last_seen_at TEXT, -- last episode where this was seen - - -- Content - text TEXT NOT NULL, -- human-readable content - embedding BLOB, -- 1536-dim float32 (text-embedding-3-small) - metadata TEXT, -- JSON: anchors (files, components, endpoints) - - -- Confidence - confidence REAL NOT NULL, - importance TEXT NOT NULL DEFAULT 'medium', -- critical | high | medium | low - - -- Lifecycle - status TEXT NOT NULL DEFAULT 'active', -- active | inactive | invalidated - valid_from TEXT NOT NULL, - valid_to TEXT, - superseded_by TEXT, - - -- Audit - user_id TEXT NOT NULL DEFAULT 'local', - created_at TEXT NOT NULL, - updated_at TEXT NOT NULL -); - --- events table -CREATE TABLE events ( - id TEXT PRIMARY KEY, - repo TEXT NOT NULL, - kind TEXT NOT NULL, -- user | assistant | tool | system - content TEXT NOT NULL, - file_paths TEXT, -- JSON array - ts TEXT NOT NULL, - processed INTEGER DEFAULT 0 -); - --- user_profile table (structured, not memories) -CREATE TABLE user_profile ( - key TEXT PRIMARY KEY, - value TEXT NOT NULL, - source TEXT NOT NULL, -- explicit | inferred - confidence REAL, - updated_at TEXT NOT NULL -); - --- memory_history table (audit trail) -CREATE TABLE memory_history ( - id TEXT PRIMARY KEY, - memory_id TEXT NOT NULL, - old_content TEXT, - new_content TEXT NOT NULL, - event TEXT NOT NULL, -- ADD | UPDATE | DELETE - created_at TEXT NOT NULL, - FOREIGN KEY (memory_id) REFERENCES memories(id) -); - --- memory_access_log table (debugging) -CREATE TABLE memory_access_log ( - id TEXT PRIMARY KEY, - memory_id TEXT NOT NULL, - access_type TEXT NOT NULL, -- search | get_context | list - query TEXT, - score REAL, - metadata TEXT, -- JSON - accessed_at TEXT NOT NULL, - FOREIGN KEY (memory_id) REFERENCES memories(id) -); -``` - -### A1.1 Declarative Key Registry - -Declarative keys are the "rigid backbone" for critical facts. Same key + different value triggers deterministic invalidation (no LLM needed). - -**Project-scoped keys** (project_id set): -``` -project.db.engine # PostgreSQL, MySQL, SQLite -project.db.version # 15, 8.0, 3.x -project.api.framework # FastAPI, Express, Rails -project.ui.framework # React, Vue, Svelte -project.language.main # Python, TypeScript, Go -project.test.command # pytest, npm test, go test -project.build.command # npm run build, cargo build -project.auth.method # JWT, session, OAuth -project.package_manager # npm, pnpm, yarn, pip, uv -project.orm # Prisma, SQLAlchemy, TypeORM -``` - -**User-scoped keys** (project_id = NULL, stored in global db): -``` -user.preferred_style # async_await, callbacks, sync -user.preferred_language # Python, TypeScript, Go -user.strict_null_checks # true, false -user.comment_style # minimal, detailed, jsdoc -user.error_handling # exceptions, result_types, errors -``` - -**Key behaviors:** -- Keys are optional - most facts remain free-text -- LLM extracts key during ingestion when pattern matches registry -- Same key + different value → deterministic invalidation of old fact -- Keys enable fast lookup without vector search - -### A1.2 Memory Lifecycle (Forget Mechanism) - -**Status values:** -- `active` - Normal, appears in retrieval -- `inactive` - Soft deleted by user (`sqrl forget`), recoverable, hidden from retrieval -- `invalidated` - Superseded by newer fact, keeps history, hidden from retrieval - -**Validity fields (for facts):** -- `valid_from` - When this became true (default: created_at) -- `valid_to` - When it stopped being true (null = still valid) -- `superseded_by` - ID of memory that replaced this - -**Retrieval filter:** -```sql -WHERE status = 'active' - AND (valid_to IS NULL OR valid_to > datetime('now')) -``` - -**Contradiction detection (during ingestion):** - -For facts with declarative `key`: -- Same key + different value → invalidate old (status='invalidated', valid_to=now, superseded_by=new_id) -- Deterministic, no LLM needed -- Example: key=project.db.engine, old value=MySQL, new value=PostgreSQL → invalidate old - -For free-text facts without key: -- LLM judges semantic conflict between new fact and similar existing facts -- High confidence conflict → invalidate old -- Low confidence → keep both, let retrieval handle via recency weighting - -**No cascade delete:** -- Invalidating a fact does NOT delete related lessons -- Related lessons get flagged in retrieval output: `dependency_changed: true` - -**CLI behavior:** -- `sqrl forget ` → status='inactive' (soft delete, recoverable) -- `sqrl forget "deprecated API"` → search + confirm + soft delete matching memories - -**v1 limitations (documented for users):** -- No TTL/auto-expiration (manual forget only) -- No hard delete/purge (data remains in SQLite file) -- Free-text contradiction detection depends on LLM, may have false positives - -### A2. Event model (`events.rs`) - -Event struct (normalized, CLI-agnostic): -- id, repo, kind (User|Assistant|Tool|System), content, file_paths, ts, processed - -Episode struct (in-memory only): -- id, repo, start_ts, end_ts, events - -Episode batching: 4-hour time window OR 50 events max (whichever first) - -### A3. Config (`config.rs`) - -Paths: -- `~/.sqrl/` (global) -- `/.sqrl/` (project) - -Files: -- `squirrel.db` - memories -- `config.toml` - settings - -Config fields: -- agents.claude_code, agents.codex_cli, agents.gemini_cli, agents.cursor (CLI selection) -- llm.provider, llm.api_key, llm.base_url -- llm.strong_model, llm.fast_model (2-tier design) -- embedding.provider, embedding.model (default: openai/text-embedding-3-small) -- daemon.idle_timeout_hours (default: 2) -- daemon.socket_path - -Projects registry (`~/.sqrl/projects.json`): -- List of initialized project paths for `sqrl sync` - ---- - -## Track B – Rust: Daemon + Watcher + IPC - -### B1. Daemon (`daemon.rs`) - -**Lazy start:** Daemon starts on first `sqrl` command, not on boot. - -**Idle shutdown:** Stop after N hours of no log activity. Flush pending Episodes on shutdown. - -Startup sequence: -1. Check if already running (socket exists and responds) -2. If not: spawn daemon process -3. Load projects registry -4. For each project, spawn watcher -5. Spawn Python Agent as child process -6. Connect via Unix socket - -### B2. Log Watchers (`watcher.rs`) - -Multi-CLI log discovery: -- `~/.claude/projects/**/*.jsonl` (Claude Code) -- `~/.codex-cli/logs/**/*.jsonl` (Codex CLI) -- `~/.gemini/logs/**/*.jsonl` (Gemini CLI) -- `~/.cursor-tutor/logs/**/*.jsonl` (Cursor) - -Line parsers for each CLI format → normalized Event - -### B3. Episode Batching - -Flush triggers: -- 50 events reached OR -- 4 hours elapsed OR -- Daemon shutdown (graceful flush) - -On flush: create Episode, send to Python via IPC, mark events processed - -### B4. IPC Client (`ipc.rs`) - -Unix socket client with JSON-RPC 2.0 protocol (MCP-compatible): -```json -{"jsonrpc": "2.0", "method": "ingest_episode", "params": {"episode": {...}}, "id": 1} -{"jsonrpc": "2.0", "result": {"memories_created": 3}, "id": 1} -``` - -Methods: -- `ingest_episode` - Process episode, extract memories -- `get_task_context` - MCP tool call -- `search_memories` - MCP tool call -- `execute_command` - CLI natural language/direct command - ---- - -## Track C – Python: Unified Agent - -### C1. Unix Socket Server (`server.py`) - -Socket at `/tmp/sqrl_agent.sock` - -Single entry point - all requests go to agent: -```python -async def handle_connection(reader, writer): - request = await read_json(reader) - result = await agent.execute(request["params"]) - await write_json(writer, {"result": result, "id": request["id"]}) -``` - -### C2. Squirrel Agent (`agent.py`) - -Single LLM-powered agent with tools using PydanticAI framework. Uses 2-tier LLM design: - -| Task | Model Tier | Default | -|------|------------|---------| -| Episode Ingestion | strong_model | gemini-2.5-pro | -| Context Compose | fast_model | gemini-3-flash | -| CLI Interpretation | fast_model | gemini-3-flash | -| Near-duplicate Check | fast_model | gemini-3-flash | - -```python -class SquirrelAgent: - def __init__(self): - self.tools = [ - # Memory tools - ingest_episode, - search_memories, - get_task_context, - forget_memory, - # Filesystem tools - find_cli_configs, - read_file, - write_file, - scan_project_logs, - # Config tools - init_project, - get_mcp_config, - set_mcp_config, - get_user_profile, - set_user_profile, - # DB tools - query_memories, - add_memory, - update_memory, - get_stats, - ] - - async def execute(self, input: dict) -> dict: - """ - Single entry point for all operations. - Agent decides which tools to use based on input. - """ - # For Episode ingestion (from daemon) - if input.get("type") == "episode": - return await self.ingest(input["episode"]) - - # For MCP calls - if input.get("type") == "mcp": - return await self.handle_mcp(input["tool"], input["args"]) - - # For CLI commands (natural language or direct) - return await self.handle_command(input.get("command", "")) -``` - -### C3. Episode Ingestion (via ingest_episode tool) - -LLM analyzes entire Episode in ONE call (segment-first approach): - -1. **Segment by kind** (not by success/failure): - - EXECUTION_TASK - coding, fixing, running commands - - PLANNING_DECISION - architecture, design choices - - RESEARCH_LEARNING - learning, exploring docs - - DISCUSSION - brainstorming, chat - -2. **For EXECUTION_TASK segments only**, classify outcome: - - SUCCESS (with evidence: tests passed, user confirmed, etc.) - - FAILURE (with evidence: error persists, user says "didn't work") - - UNCERTAIN (no clear evidence - conservative default) - -3. **Extract memories based on segment kind:** - - EXECUTION_TASK → lesson (with outcome), fact (knowledge discovered) - - PLANNING_DECISION → fact (decisions), lesson (rationale for rejected options), profile - - RESEARCH_LEARNING → fact (knowledge), lesson (key learnings) - - DISCUSSION → profile (user preferences), lesson (insights) - -4. **Detect user frustration signals** and boost importance: - - Swear words, angry language → `importance: critical` - - Repeated complaints ("again", "still broken") → `importance: high` - - Mild frustration → `importance: medium` (default) - - Add `metadata.user_frustration`: none | mild | moderate | severe - - Frustration signals make associated failure lessons higher priority in retrieval - -5. **Contradiction check for facts:** - - Extract semantic_key if possible (db.engine, api.framework, etc.) - - Check existing facts with same key → invalidate old if conflict - - Free-text facts → LLM judges semantic conflict - -6. **Near-duplicate check** before ADD (0.9 similarity threshold) - -UUID→integer mapping when showing existing memories to LLM (prevents hallucination). - -### C4. Schemas (`schemas/`) - -**Memory schema:** -- id, project_id (NULL for global), memory_type (lesson | fact | profile) -- text (human-readable content), embedding (1536-dim) -- metadata (JSON: anchors - files, components, endpoints) -- confidence, importance (critical | high | medium | low) -- user_id, created_at, updated_at - -**Lesson-specific fields:** -- outcome: success | failure | uncertain - -**Fact-specific fields:** -- fact_type: knowledge | process (optional) -- key: declarative key (project.db.engine, user.preferred_style, etc.) -- value: declarative value (PostgreSQL, async_await, etc.) -- evidence_source: success | failure | neutral | manual -- support_count: number of episodes that support this fact -- last_seen_at: timestamp of last episode where seen - -**Lifecycle fields (all types):** -- status: active | inactive | invalidated -- valid_from: timestamp (when this became true) -- valid_to: timestamp | null (when it stopped being true) -- superseded_by: memory_id | null (for invalidated facts) - -**UserProfile schema (structured identity, not memories):** -- key, value, source (explicit|inferred), confidence, updated_at -- Examples: name, role, experience_level, company, primary_use_case - ---- - -## Track D – Python: Tools Implementation - -### D1. Memory Tools (`tools/memory.py`) - -**ingest_episode(events):** LLM analysis, task segmentation, outcome classification, memory extraction - -**search_memories(query, filters):** Embed query, sqlite-vec search, return ranked results - -**get_task_context(task, budget):** -1. Vector search retrieves top 20 candidates -2. LLM (fast_model) reranks + composes context prompt: - - Selects relevant memories - - Resolves conflicts between memories - - Merges related memories - - Generates structured prompt with memory IDs -3. Returns ready-to-inject context prompt within token budget - -**Context output structure:** -```json -{ - "project_facts": [ - {"key": "project.db.engine", "value": "PostgreSQL", "text": "..."}, - {"key": "project.api.framework", "value": "FastAPI", "text": "..."} - ], - "user_prefs": [ - {"key": "user.preferred_style", "value": "async_await", "text": "..."} - ], - "lessons": [ - {"outcome": "failure", "text": "Validate user_id before DB insert...", "id": "mem_123"}, - {"outcome": "success", "text": "Use repository pattern for DB access...", "id": "mem_456"} - ], - "process_facts": [ - {"text": "Auth module handles JWT validation in middleware.", "id": "mem_789"} - ], - "profile": { - "name": "Alice", - "role": "Backend Developer", - "experience_level": "Senior" - } -} -``` - -**forget_memory(id_or_query):** -- If ID: set status='inactive' (soft delete, recoverable) -- If natural language query: search → confirm with user → soft delete matches - -**export_memories(filters, format):** Export memories as JSON for sharing/backup - -**import_memories(data):** Import memories from JSON - -### D2. Filesystem Tools (`tools/filesystem.py`) - -**find_cli_configs():** Scan for ~/.claude, ~/.codex-cli, ~/.gemini, etc. - -**scan_project_logs(project_root, token_limit):** Find logs mentioning project files, return within token limit - -**read_file(path), write_file(path, content):** Basic file operations for config management - -### D3. Config Tools (`tools/config.py`) - -**init_project(path, skip_history):** -1. Create `/.sqrl/squirrel.db` -2. If not skip_history: scan_project_logs → ingest -3. For each enabled CLI in `config.agents`: - - Configure MCP (add Squirrel server to CLI's MCP config) - - Inject instruction text to agent file (CLAUDE.md, AGENTS.md, GEMINI.md, .cursor/rules/) -4. Register project in `~/.sqrl/projects.json` - -**sync_projects():** -1. Read enabled CLIs from `config.agents` -2. For each project in `~/.sqrl/projects.json`: - - For each enabled CLI not yet configured in that project: - - Configure MCP - - Inject instruction text - -**get_mcp_config(cli), set_mcp_config(cli, server, config):** Read/write MCP config files - -**get_agent_instructions(cli), set_agent_instructions(cli, content):** Read/write agent instruction files - -**get_user_profile(), set_user_profile(key, value):** Manage user_profile table - -### D3.1 MCP Config Locations - -| CLI | MCP Config Location | -|-----|---------------------| -| Claude Code | `~/.claude.json` or `/.mcp.json` | -| Codex CLI | `codex mcp add-server` command | -| Gemini CLI | `/.gemini/settings.json` | -| Cursor | `~/.cursor/mcp.json` or `/.cursor/mcp.json` | - -MCP server definition: -```json -{ - "mcpServers": { - "squirrel": { - "command": "sqrl-daemon", - "args": ["--mcp"], - "disabled": false - } - } -} -``` - -### D3.2 Agent Instruction Files - -| CLI | Instruction File | -|-----|------------------| -| Claude Code | `/CLAUDE.md` | -| Codex CLI | `/AGENTS.md` | -| Gemini CLI | `/GEMINI.md` | -| Cursor | `/.cursor/rules/squirrel.mdc` | - -Instruction text to inject: -```markdown -## Squirrel Memory System - -This project uses Squirrel for persistent memory across sessions. - -ALWAYS call `squirrel_get_task_context` BEFORE: -- Fixing bugs (to check if this bug was seen before) -- Refactoring code (to get patterns that worked/failed) -- Adding features touching existing modules -- Debugging errors that seem familiar - -DO NOT call for: -- Simple typo fixes -- Adding comments -- Formatting changes -``` - -### D4. DB Tools (`tools/db.py`) - -**query_memories(filters):** Direct DB query with filtering - -**add_memory(memory):** Insert + log to history - -**update_memory(id, changes):** Update + log old/new to history - -**get_stats():** Memory counts, access stats, etc. - -### D5. Embeddings (`embeddings.py`) - -API-based embeddings via OpenAI (text-embedding-3-small, 1536-dim). - -Supports multiple providers via config. Batch embedding with retry logic. - -### D6. Retrieval (`retrieval.py`) - -Similarity search via sqlite-vec. - -Scoring: `w_sim * similarity + w_imp * importance_weight + w_rec * recency + w_frust * frustration_boost` -- importance_weight: critical=1.0, high=0.75, medium=0.5, low=0.25 -- frustration_boost: severe=0.3, moderate=0.2, mild=0.1, none=0.0 - -Frustration-flagged memories surface earlier to prevent recurring pain points. - -Access logging to memory_access_log table. - ---- - -## Track E – MCP + CLI - -### E1. MCP Server (`mcp.rs`) - -Uses `rmcp` (official MCP SDK from modelcontextprotocol/rust-sdk). - -2 tools: -``` -squirrel_get_task_context - - project_root: string (required) - - task: string (required) - - context_budget_tokens: integer (default: 400) - -squirrel_search_memory - - project_root: string (required) - - query: string (required) - - top_k: integer (default: 10) -``` - -For trivial queries, return empty fast (<20ms). - -### E2. CLI (`cli.rs`) - -Thin shell that passes to Python agent: - -```rust -fn main() { - ensure_daemon_running()?; - - let args: Vec = std::env::args().skip(1).collect(); - let input = args.join(" "); - - let response = ipc_client.send("agent_execute", { - "type": "command", - "command": input - }); - - println!("{}", response); -} -``` - -Supports both natural language and direct commands: -- `sqrl "setup this project"` → agent interprets -- `sqrl init --skip-history` → agent interprets -- `sqrl config` → interactive CLI selection -- `sqrl sync` → update all projects with new CLI configs -- `sqrl update` → auto-update via axoupdater -- `sqrl export ` → export memories as JSON -- `sqrl import ` → import memories - ---- - -## Phase X – Hardening - -### Logging & Observability -- Structured logging (Rust: `tracing`, Python: `structlog`) -- Metrics: events/episodes/memories processed, latency - -### Testing -- Unit tests: storage, events, agent tools -- Integration tests: full flow from log to memory to retrieval - -### Build & Release (dist) - -Uses `dist` (cargo-dist) as single release orchestrator: -- Builds Rust daemon for Mac/Linux/Windows -- Builds Python agent via PyInstaller (as dist workspace member) -- Generates installers: Homebrew, MSI, shell/powershell scripts - -### Cross-Platform Installation - -| Platform | Primary | Fallback | -|----------|---------|----------| -| Mac | `brew install sqrl` | install script | -| Linux | `brew install sqrl` | install script, AUR, nixpkg | -| Windows | MSI installer | winget, install script | - -Windows note: MSI recommended over raw .exe to reduce SmartScreen/AV friction. - -### Auto-Update (axoupdater) - -- `sqrl update` uses axoupdater (dist's official updater) -- Updates both Rust daemon and Python agent together -- Reads dist install receipt to determine installed version/source - ---- - -## Timeline Summary - -| Week | Track A | Track B | Track C | Track D | Track E | -|------|---------|---------|---------|---------|---------| -| 0 | Scaffold | Scaffold | Scaffold | - | - | -| 1 | Storage, Events, Config | Daemon start | Agent skeleton | - | - | -| 2 | - | Watchers, IPC, Batching | - | Memory tools | - | -| 3 | - | Integration | - | Filesystem, Config tools | MCP server | -| 4 | - | - | - | DB tools, Retrieval | CLI | -| 5+ | - | Hardening | - | Hardening | Cross-platform | - ---- - -## Team Assignment (3 developers) - -**Developer 1 (Rust focus):** -- Phase 0: Rust scaffold -- Track A: All -- Track B: All -- Track E: MCP server, CLI - -**Developer 2 (Python focus):** -- Phase 0: Python scaffold -- Track C: All -- Track D: All - -**Developer 3 (Full-stack / Integration):** -- Phase 0: CI, docs -- Agent prompts -- Cross-platform packaging -- Phase X: Testing, documentation - ---- - -## v1 Scope - -- Passive log watching (4 CLIs) -- Episode segmentation (EXECUTION_TASK / PLANNING_DECISION / RESEARCH_LEARNING / DISCUSSION) -- Success detection for EXECUTION_TASK only (SUCCESS/FAILURE/UNCERTAIN with evidence) -- Unified Python agent with tools -- Natural language CLI -- MCP integration (2 tools) -- Lazy daemon (start on demand, stop after 2hr idle) -- Retroactive log ingestion on init (token-limited) -- 3 memory types (lesson, fact, profile) with scope flag -- Declarative keys for facts (project.* and user.*) with deterministic conflict detection -- Evidence source tracking for facts (success/failure/neutral/manual) -- Memory lifecycle: status (active/inactive/invalidated) + validity (valid_from/valid_to) -- Fact contradiction detection (declarative key match + LLM for free-text) -- Soft delete (`sqrl forget`) - no hard purge -- Near-duplicate deduplication (0.9 threshold) -- Frustration detection (swear words, anger → boost importance, prioritize in retrieval) -- Cross-platform (Mac, Linux, Windows) -- Export/import memories (JSON) -- Auto-update (`sqrl update`) -- Memory consolidation -- Retrieval debugging tools -- CLI selection (`sqrl config`) + MCP wiring + agent instruction injection -- `sqrl sync` for updating existing projects with new CLIs - -**v1 limitations:** -- No TTL/auto-expiration (manual forget only) -- No hard delete (soft delete only, data remains in SQLite) -- Free-text contradiction detection may have false positives - -## v2 Scope (Future) - -- Team/cloud sync (group.db, share command, team management) -- Deep CLI integrations (Claude Code hooks, Cursor extension) -- Team analytics dashboard -- Memory marketplace -- TTL / temporary memory (auto-expiration) -- Hard purge for privacy/compliance -- Memory linking + evolution (A-MEM style) -- Richer conflict detection with schema/key registry -- `get_memory_history` API for debugging invalidation chains - ---- - -## v2 Architecture: Team/Cloud - -### Overview - -v2 adds team memory sharing via `group.db` - a separate database that syncs with cloud. Individual memories stay in `squirrel.db` (local-only), team memories go to `group.db` (synced). - -### 3-Layer Database Architecture (v2) - -| Layer | DB File | Contents | Sync | -|-------|---------|----------|------| -| **Global** | `~/.sqrl/squirrel.db` | lesson, fact, profile (scope=global) | Local only | -| **Project** | `/.sqrl/squirrel.db` | lesson, fact (scope=project) | Local only | -| **Team** | `~/.sqrl/group.db` + `/.sqrl/group.db` | Shared memories (owner=team) | Cloud | - -### Memory Schema (v2) - -Additional fields for team support: - -```sql -CREATE TABLE memories ( - -- ... all v1 fields ... - owner TEXT NOT NULL DEFAULT 'individual', -- individual | team - team_id TEXT, -- team identifier (for owner=team) - contributed_by TEXT, -- user who shared (for owner=team) - source_memory_id TEXT -- original memory ID (if promoted to team) -); -``` - -### Team Tools (v2) - -**share_memory(memory_id):** Promote individual memory to team -1. Read from `squirrel.db` -2. Copy to `group.db` with `owner: team` -3. Set `contributed_by`, `source_memory_id` -4. Sync triggers cloud upload - -**team_join(team_id), team_leave():** Team membership management - -**team_export(filters):** Export team memories for offline/backup - -### Sync Architecture - -**Local-first with background sync:** -- `group.db` is local copy, always available -- Background process syncs with cloud -- Users never wait for network -- Conflict resolution: last-write-wins with vector clocks - -**Scaling considerations (from research):** -- Individual user (6 months): ~6MB (900 memories) -- Team (10,000 users): ~6GB if full sync - NOT viable - -**Hybrid approach for large teams:** -| Team Size | Strategy | -|-----------|----------| -| Small (<100) | Full sync - all team memories in local group.db | -| Medium (100-1000) | Partial sync - recent + relevant memories locally | -| Large (1000+) | Cloud-primary - query cloud, cache locally | - -**Reference:** Figma, Notion, Linear all use server-first or partial sync. Nobody syncs everything locally at scale. - -### Team Commands (v2) - -```bash -sqrl team join # Join team, start syncing group.db -sqrl team leave # Leave team, remove group.db -sqrl share # Promote individual memory to team -sqrl share --all # Share all individual memories to team -sqrl team export # Export team memories to local -``` - -### Migration Paths - -**Local → Cloud (user subscribes):** -```bash -sqrl share --all # Promotes all individual memories to team -``` - -**Cloud → Local (team exports):** -```bash -sqrl team export --project # Downloads team memories to local squirrel.db -``` - -### Config (v2) - -```toml -# ~/.sqrl/config.toml -[team] -id = "abc-team-id" -sync_interval_seconds = 300 # 5 min background sync -sync_strategy = "full" # full | partial | cloud-primary -``` - -### Retrieval (v2) - -Context retrieval queries BOTH databases: -1. `squirrel.db` (individual memories) -2. `group.db` (team memories) -3. LLM reranks combined results - -Team memories get attribution: "From team member Alice" - ---- - -## Patterns from Competitor Analysis - -| Pattern | Source | Location | -|---------|--------|----------| -| UUID→integer mapping for LLM | mem0 | Agent ingest | -| History tracking (old/new content) | mem0 | memory_history table | -| Structured exceptions | mem0 | All tools | -| Soft-delete (state column) | mem0 | memories table | -| Access logging | mem0 | memory_access_log table | -| Success detection | claude-cache | Agent ingest | -| Pitfall learning | claude-cache | Memory types | -| Unified agent with tools | letta | Agent architecture | -| Session Q&A tracking | cognee | memory_access_log | - -**Key insight:** Passive learning requires success detection. We let the LLM decide task outcomes instead of building a rules engine. diff --git a/EXAMPLE.md b/EXAMPLE.md deleted file mode 100644 index 17ef83d..0000000 --- a/EXAMPLE.md +++ /dev/null @@ -1,723 +0,0 @@ -# Squirrel: Complete Process Walkthrough - -Detailed example demonstrating the entire Squirrel data flow from installation to personalized AI context. - -## Core Concept - -Squirrel watches AI tool logs, groups events into **Episodes** (4-hour time windows), and sends them to a unified Python Agent for analysis: - -1. **Segment by Kind** - Not all sessions are coding tasks. Identify segment type first: - - `EXECUTION_TASK` - coding, fixing bugs, running commands - - `PLANNING_DECISION` - architecture, design, tech choices - - `RESEARCH_LEARNING` - learning, exploring docs - - `DISCUSSION` - brainstorming, chat -2. **Classify Outcomes** - Only for EXECUTION_TASK: SUCCESS | FAILURE | UNCERTAIN (with evidence) -3. **Detect Frustration** - Swear words, anger → boost memory importance -4. **Extract Memories** - Based on segment kind, not just success/failure - -Episode = batch of events from same repo within 4-hour window (internal batching, not a product concept). - -**The key insight:** Not every session is a "task" with success/failure. Architecture discussions, research, and chat produce valuable memories without outcomes. We segment first, then extract appropriately. - ---- - -## Scenario - -Developer "Alice" working on `inventory-api` (FastAPI project). - -Alice's coding preferences: -- Type hints everywhere -- pytest with fixtures -- Async/await patterns - ---- - -## Phase 1: Installation & Setup - -### Step 1.1: Install Squirrel - -```bash -# Universal install (Mac/Linux/Windows) -curl -sSL https://sqrl.dev/install.sh | sh - -# Or platform-specific -brew install sqrl # Mac -winget install sqrl # Windows -``` - -### Step 1.2: CLI Selection (First Run) - -```bash -sqrl config -# Interactive prompt: select which CLIs you use -# → Claude Code: yes -# → Codex CLI: yes -# → Gemini CLI: no -# → Cursor: yes -``` - -This stores CLI selection in `~/.sqrl/config.toml`: -```toml -[agents] -claude_code = true -codex_cli = true -gemini_cli = false -cursor = true -``` - -### Step 1.3: Project Initialization - -```bash -cd ~/projects/inventory-api -sqrl init -``` - -What happens: -1. First `sqrl` command auto-starts daemon (lazy start) -2. `sqrl init` triggers agent via IPC -3. Creates `.sqrl/squirrel.db` for project memories -4. Agent scans for CLI log folders containing this project -5. Agent asks: ingest historical logs? (token-limited, not time-limited) -6. For each enabled CLI (from `config.agents`): - - Configures MCP (adds Squirrel server to CLI's MCP config) - - Injects instruction text to agent file (CLAUDE.md, AGENTS.md, .cursor/rules/) -7. Registers project in `~/.sqrl/projects.json` - -File structure after init: -``` -~/.sqrl/ -├── config.toml # API keys, settings, CLI selection -├── squirrel.db # Global memories (lesson, fact, profile with scope=global) -├── projects.json # List of initialized projects (for sqrl sync) -└── logs/ # Daemon logs - -~/projects/inventory-api/ -├── .sqrl/ -│ └── squirrel.db # Project memories (lesson, fact with scope=project) -├── CLAUDE.md # ← Squirrel instructions injected -└── AGENTS.md # ← Squirrel instructions injected -``` - -### Step 1.4: Agent Instruction Injection - -For each enabled CLI, Squirrel adds this block to the agent instruction file: - -```markdown -## Squirrel Memory System - -This project uses Squirrel for persistent memory across sessions. - -ALWAYS call `squirrel_get_task_context` BEFORE: -- Fixing bugs (to check if this bug was seen before) -- Refactoring code (to get patterns that worked/failed) -- Adding features touching existing modules -- Debugging errors that seem familiar - -DO NOT call for: -- Simple typo fixes -- Adding comments -- Formatting changes -``` - -This increases the probability that AI tools will call Squirrel MCP tools at the right moments. - -### Step 1.5: Syncing New CLIs - -Weeks later, Alice enables Cursor globally: - -```bash -sqrl config # select Cursor - -# Update all existing projects -sqrl sync -# → Adds MCP config + instructions for Cursor to all registered projects -``` - -### Step 1.6: Natural Language CLI - -The agent handles all CLI commands: - -```bash -sqrl "what do you know about auth here" -sqrl "show my coding style" -sqrl "I prefer functional programming" -sqrl "configure gemini cli to use squirrel" -sqrl "forget the memory about deprecated API" -``` - -Or direct commands: -```bash -sqrl search "database patterns" -sqrl status -sqrl config set llm.model claude-sonnet -sqrl sync # Update all projects with new CLI configs -``` - ---- - -## Phase 2: Learning (Passive Collection) - -### Step 2.1: Alice Codes (Two Tasks in One Session) - -``` -# Task 1: Add endpoint (SUCCESS) -Alice: "Add a new endpoint to get inventory items by category" -Claude Code: "I'll create a GET endpoint..." -Alice: "Use async def, add type hints, and write a pytest fixture" -Claude Code: [Revises code with async, types, fixture] -Alice: "Perfect, tests pass!" - -# Task 2: Fix auth bug (FAILURE then SUCCESS, with frustration) -Alice: "There's an auth loop bug when tokens expire" -Claude Code: "Let me check localStorage..." -[Error persists] -Alice: "Still broken" -Claude Code: "Let me try checking cookies..." -[Error persists] -Alice: "This is so frustrating, we've been going in circles!" -Claude Code: "I think the issue is in useEffect cleanup..." -[Implements fix] -Alice: "That fixed it, thanks!" -``` - -### Step 2.2: Rust Daemon Watches Logs - -Daemon watches log files from all supported CLIs: -``` -~/.claude/projects/**/*.jsonl # Claude Code -~/.codex-cli/logs/**/*.jsonl # Codex CLI -~/.gemini/logs/**/*.jsonl # Gemini CLI -``` - -Parses into normalized Events: - -```rust -let event = Event { - id: "evt_001", - repo: "/Users/alice/projects/inventory-api", - kind: "user", // user | assistant | tool | system - content: "Use async def, add type hints...", - file_paths: vec![], - ts: "2025-11-25T10:01:00Z", - processed: false, -}; -storage.save_event(event)?; -``` - -### Step 2.3: Episode Batching - -Episodes flush on **4-hour time window** OR **50 events max** (whichever comes first): - -```rust -fn should_flush_episode(buffer: &EventBuffer) -> bool { - let window_hours = 4; - let max_events = 50; - - buffer.events.len() >= max_events || - buffer.oldest_event_age() >= Duration::hours(window_hours) -} - -// Flush triggers IPC call to Python Agent -fn flush_episode(repo: &str, events: Vec) { - let episode = Episode { - id: generate_uuid(), - repo: repo.to_string(), - start_ts: events.first().ts, - end_ts: events.last().ts, - events: events, - }; - - ipc_client.send(json!({ - "method": "ingest_episode", - "params": { "episode": episode }, - "id": 1 - })); -} -``` - ---- - -## Phase 3: Memory Extraction (Python Agent) - -### Step 3.1: Agent Analyzes Episode (Segment-First Approach) - -The unified agent receives the Episode and uses segment-first analysis: - -```python -async def ingest_episode(episode: dict) -> dict: - # Build context from events - context = "\n".join([ - f"[{e['kind']}] {e['content']}" - for e in episode["events"] - ]) - - # LLM analyzes: segments first, then memories in ONE call - response = await llm.call(INGEST_PROMPT.format(context=context)) - - return { - "segments": [ - { - "id": "seg_1", - "kind": "EXECUTION_TASK", - "title": "Add category endpoint", - "event_range": [0, 4], - "outcome": { - "status": "SUCCESS", - "evidence": ["User said 'Perfect, tests pass!'"] - } - }, - { - "id": "seg_2", - "kind": "EXECUTION_TASK", - "title": "Fix auth loop bug", - "event_range": [5, 10], - "outcome": { - "status": "SUCCESS", - "evidence": ["User said 'That fixed it, thanks!'"] - }, - "user_frustration": "moderate" - } - ], - "memories": [ - { - "memory_type": "fact", - "scope": "global", - "key": "user.preferred_style", - "value": "async_await", - "text": "Prefers async/await with type hints for all handlers", - "evidence_source": "success", - "source_segments": ["seg_1"], - "confidence": 0.9 - }, - { - "memory_type": "lesson", - "outcome": "failure", - "scope": "project", - "text": "Auth token refresh loops are NOT caused by localStorage or cookies - check useEffect cleanup first", - "importance": "high", - "metadata": {"user_frustration": "moderate"}, - "source_segments": ["seg_2"], - "confidence": 0.9 - }, - { - "memory_type": "lesson", - "outcome": "success", - "scope": "project", - "text": "For auth redirect loops, fix useEffect cleanup to prevent re-triggering on token refresh", - "source_segments": ["seg_2"], - "confidence": 0.9 - }, - { - "memory_type": "fact", - "scope": "project", - "text": "Tried localStorage fix (failed), tried cookies fix (failed), useEffect cleanup fix worked", - "evidence_source": "neutral", - "source_segments": ["seg_2"], - "confidence": 0.85 - } - ] - } -``` - -INGEST Prompt: -``` -Analyze this coding session (~4 hours of activity): - -[user] Add a new endpoint to get inventory items by category -[assistant] I'll create a GET endpoint... -[user] Use async def, add type hints, and write a pytest fixture -[assistant] [Revises code with async, types, fixture] -[user] Perfect, tests pass! -[user] There's an auth loop bug when tokens expire -[assistant] Let me check localStorage... -[user] Still broken -[assistant] Let me try checking cookies... -[user] This is so frustrating, we've been going in circles! -[assistant] I think the issue is in useEffect cleanup... -[user] That fixed it, thanks! - -Analyze this session using SEGMENT-FIRST approach: - -1. SEGMENT the episode by kind (each segment = 5-20 events): - - EXECUTION_TASK: coding, fixing, running commands (CAN have outcome) - - PLANNING_DECISION: architecture, design (has resolution: DECIDED/OPEN) - - RESEARCH_LEARNING: learning, exploring (has resolution: ANSWERED/PARTIAL) - - DISCUSSION: brainstorming, chat (no outcome) - -2. For EXECUTION_TASK segments ONLY, classify outcome: - - SUCCESS: with evidence (tests passed, user confirmed, etc.) - - FAILURE: with evidence (error persists, user says "didn't work") - - UNCERTAIN: no clear evidence (conservative default) - - IMPORTANT: Other segment kinds NEVER have SUCCESS/FAILURE. - -3. Detect user frustration signals: - - Swear words, strong anger → importance: critical, user_frustration: severe - - Repeated complaints ("again", "still broken") → importance: high, user_frustration: moderate - - Mild frustration → importance: medium, user_frustration: mild - - No frustration → importance: medium, user_frustration: none - -4. Extract memories based on segment kind: - - EXECUTION_TASK: lesson (with outcome), fact (knowledge discovered) - - PLANNING_DECISION: fact (decisions), lesson (rationale), profile - - RESEARCH_LEARNING: fact (knowledge), lesson (learnings) - - DISCUSSION: profile (preferences), lesson (insights) - -Return segments[] and memories[] with source_segment references. -``` - -### Step 3.2: Near-Duplicate Check + Save Memory - -Agent uses its DB tools to check for duplicates: - -```python -# Before saving, check for near-duplicates -candidates = await db_tools.search_similar( - repo=memory.repo, - content=memory.content, - memory_types=[memory.memory_type], - top_k=5 -) -for candidate in candidates: - if candidate.similarity >= 0.9: - # Near-duplicate found - merge or skip - return await merge_or_skip(memory, candidate) - -# No duplicate, save new memory -await db_tools.add_memory(memory) -``` - ---- - -## Phase 4: Context Retrieval (MCP) - -### Step 4.1: MCP Tool Call - -Claude Code (or other AI tool) calls Squirrel via MCP: - -```json -{ - "tool": "squirrel_get_task_context", - "arguments": { - "project_root": "/Users/alice/projects/inventory-api", - "task": "Add a delete endpoint for inventory items", - "context_budget_tokens": 400 - } -} -``` - -### Step 4.2: Vector Search (Candidate Retrieval) - -The agent receives the MCP request via IPC and retrieves candidates: - -```python -async def get_task_context(project_root: str, task: str, budget: int) -> dict: - # For trivial queries, return empty fast (<20ms) - if is_trivial_task(task): # "fix typo", "add comment" - return {"context_prompt": "", "memory_ids": [], "tokens_used": 0} - - # Vector search retrieves top 20 candidates from both DBs - candidates = await retrieval.search( - task=task, - project_root=project_root, - include_global=True, # user_style from global DB - top_k=20 - ) - # candidates now contains ~20 memories ranked by embedding similarity -``` - -### Step 4.3: LLM Rerank + Compose (fast_model) - -LLM reranks candidates and composes a context prompt in ONE call: - -```python - # LLM reranks + composes context prompt (uses fast_model) - response = await llm.call( - model=config.fast_model, # gemini-3-flash - prompt=COMPOSE_PROMPT.format( - task=task, - candidates=format_candidates(candidates), - budget=budget - ) - ) - - return { - "context_prompt": response.prompt, # Ready-to-inject text - "memory_ids": response.selected_ids, # For tracing - "tokens_used": count_tokens(response.prompt) - } -``` - -COMPOSE_PROMPT: -``` -Task: {task} - -Candidate memories (ranked by similarity): -{candidates} - -Select the most relevant memories for this task. Then compose a context prompt that: -1. Prioritizes lessons with outcome=failure (what NOT to do) first -2. Includes relevant lessons (outcome=success) and facts -3. Resolves conflicts between memories (newer wins) -4. Merges related memories to save tokens -5. Stays within {budget} tokens - -Return: -- selected_ids: list of memory IDs you selected -- prompt: the composed context prompt for the AI tool -``` - -### Step 4.4: Response - -```json -{ - "context_prompt": "## Context from Squirrel\n\n**Style Preferences:**\n- Use async/await with type hints for all handlers [mem_abc123]\n\n**Project Facts:**\n- This project uses FastAPI with Pydantic models [mem_def456]\n\n**Relevant for this task:** You're adding an HTTP endpoint, so follow the async pattern and define a Pydantic response model.", - "memory_ids": ["mem_abc123", "mem_def456"], - "tokens_used": 89 -} -``` - -The AI tool injects this `context_prompt` directly into its system prompt for better responses. - ---- - -## Phase 5: Daemon Lifecycle - -### Lazy Start -``` -User runs: sqrl search "auth" - ↓ -CLI checks if daemon running (Unix socket) - ↓ -Not running → CLI starts daemon in background - ↓ -Daemon starts → CLI connects → command executed -``` - -### Idle Shutdown -``` -Daemon tracks last activity timestamp - ↓ -Every minute: check if idle > 2 hours - ↓ -If idle: flush any pending Episodes → graceful shutdown - ↓ -Next sqrl command → daemon starts again -``` - -No manual daemon management. No system services. Just works. - ---- - -## Data Schema - -### Event (normalized from all CLIs) - -```sql -CREATE TABLE events ( - id TEXT PRIMARY KEY, - repo TEXT NOT NULL, - kind TEXT NOT NULL, -- user | assistant | tool | system - content TEXT NOT NULL, - file_paths TEXT, -- JSON array - ts TEXT NOT NULL, - processed INTEGER DEFAULT 0 -); -``` - -### Memory (squirrel.db) - -```sql -CREATE TABLE memories ( - id TEXT PRIMARY KEY, - project_id TEXT, -- NULL for global/user-scope memories - memory_type TEXT NOT NULL, -- lesson | fact | profile - - -- For lessons (task-level patterns/pitfalls) - outcome TEXT, -- success | failure | uncertain (lesson only) - - -- For facts - fact_type TEXT, -- knowledge | process (optional) - key TEXT, -- declarative key: project.db.engine, user.preferred_style - value TEXT, -- declarative value: PostgreSQL, async_await - evidence_source TEXT, -- success | failure | neutral | manual (fact only) - support_count INTEGER, -- approx episodes that support this fact - last_seen_at TEXT, -- last episode where this was seen - - -- Content - text TEXT NOT NULL, -- human-readable content - embedding BLOB, -- 1536-dim float32 (text-embedding-3-small) - metadata TEXT, -- JSON: anchors (files, components, endpoints) - - -- Confidence - confidence REAL NOT NULL, - importance TEXT NOT NULL DEFAULT 'medium', -- critical | high | medium | low - - -- Lifecycle - status TEXT NOT NULL DEFAULT 'active', -- active | inactive | invalidated - valid_from TEXT NOT NULL, - valid_to TEXT, - superseded_by TEXT, - - -- Audit - user_id TEXT NOT NULL DEFAULT 'local', - created_at TEXT NOT NULL, - updated_at TEXT NOT NULL -); -``` - -**Status values:** -- `active` - Normal, appears in retrieval -- `inactive` - Soft deleted via `sqrl forget`, hidden but recoverable -- `invalidated` - Superseded by newer fact, hidden but keeps history - -**Declarative key examples:** -- `project.db.engine` → PostgreSQL, MySQL, SQLite -- `project.api.framework` → FastAPI, Express, Rails -- `user.preferred_style` → async_await, callbacks, sync -- `user.comment_style` → minimal, detailed, jsdoc - -Same key + different value → deterministic invalidation of old fact. - -### User Profile (structured identity) - -```sql -CREATE TABLE user_profile ( - user_id TEXT PRIMARY KEY, - name TEXT, - role TEXT, - experience_level TEXT, - company TEXT, - primary_use_case TEXT, - created_at TEXT NOT NULL, - updated_at TEXT NOT NULL -); -``` - ---- - -## Summary - -| Phase | What Happens | -|-------|--------------| -| Install | Universal script or package manager | -| CLI Selection | `sqrl config` - select which CLIs you use (Claude Code, Codex, etc.) | -| Init | Creates DB, ingests history, configures MCP + injects agent instructions for enabled CLIs | -| Sync | `sqrl sync` - updates all projects when new CLIs enabled | -| Learning | Daemon watches CLI logs, parses to Events | -| Batching | Groups events into Episodes (4hr OR 50 events) | -| **Segmentation** | Agent segments by kind: EXECUTION_TASK / PLANNING_DECISION / RESEARCH_LEARNING / DISCUSSION | -| **Outcome** | For EXECUTION_TASK only: SUCCESS/FAILURE/UNCERTAIN (with evidence) | -| **Frustration** | Detects anger/swearing → boosts importance (critical/high), stores user_frustration in metadata | -| Extraction | Based on segment kind: lesson (with outcome), fact (with key/evidence_source), profile | -| **Declarative Keys** | Facts with project.* or user.* keys enable deterministic conflict detection | -| **Contradiction** | Same key + different value → old fact invalidated (no LLM); free-text → LLM judges | -| Dedup | Near-duplicate check (0.9 similarity) before ADD | -| Retrieval | MCP → Vector search (top 20) → LLM reranks + composes context prompt | -| Forget | `sqrl forget` → soft delete (status=inactive), recoverable | -| Idle | 2hr no activity → daemon stops, next command restarts | - -### Why Segment-First Matters - -Not all sessions are coding tasks with success/failure: -- Architecture discussions → produce decisions (fact), not outcomes -- Research sessions → produce knowledge (fact), not outcomes -- Brainstorming → produces insights (lesson) and preferences (profile) - -Segment-first ensures we extract appropriate memories from each session type, and only apply SUCCESS/FAILURE to actual execution tasks. - -### Why Contradiction Detection Matters - -Facts change over time: -- Day 1: "Project uses PostgreSQL" (fact) -- Day 30: "Migrated to MySQL" (new fact) - -Contradiction detection auto-invalidates old facts when new conflicting facts arrive, keeping retrieval clean and accurate. - ---- - -## Key Design Decisions - -| Decision | Choice | Why | -|----------|--------|-----| -| **Unified Agent** | Single Python agent with tools | One LLM brain for all operations | -| **2-tier LLM** | strong_model + fast_model | Pro for complex reasoning, Flash for quick tasks | -| **Lazy Daemon** | Start on command, stop after 2hr idle | No system service complexity | -| Episode trigger | 4-hour window OR 50 events | Balance context vs LLM cost | -| **Segment-first** | Segment by kind before outcome classification | Not all sessions are tasks with outcomes | -| **Segment kinds** | EXECUTION_TASK / PLANNING / RESEARCH / DISCUSSION | Different session types produce different memories | -| **Outcome only for EXECUTION_TASK** | SUCCESS/FAILURE/UNCERTAIN with evidence | Avoid classifying discussions as "failures" | -| **Frustration detection** | Anger/swearing → importance boost + metadata flag | High-pain failures get prioritized in retrieval | -| Memory extraction | Based on segment kind | Architecture produces facts, coding produces lessons | -| **Declarative keys** | project.* and user.* keys for facts | Deterministic conflict detection (no LLM) | -| **Evidence source** | success/failure/neutral/manual on facts | Track how a fact was learned | -| **Memory lifecycle** | status (active/inactive/invalidated) + validity | Soft delete + contradiction handling | -| **Fact contradiction** | Declarative key match + LLM for free-text | Auto-invalidate old when new conflicts | -| **Soft delete only (v1)** | `sqrl forget` → status=inactive | Recoverable, no hard purge until v2 | -| **Context compose** | LLM reranks + generates prompt (fast_model) | Better than math scoring, one call | -| **Natural language CLI** | Thin shell passes to agent | "By the way" - agent handles all | -| **Retroactive ingestion** | Token-limited, not time-limited | Fair for all project sizes | -| User profile | Separate table (structured identity) | name, role, experience_level - not learned | -| **2-layer DB** | Global (squirrel.db) + Project (squirrel.db) | Scope-based separation | -| **CLI selection** | User picks CLIs in `sqrl config` | Only configure what user actually uses | -| **Agent instruction injection** | Add Squirrel block to CLAUDE.md, AGENTS.md, etc. | Increase MCP call success rate | -| **sqrl sync** | Update all projects when new CLI enabled | User stays in control, no magic patching | -| Near-duplicate threshold | 0.9 similarity | Avoid redundant memories | -| Trivial query fast-path | Return empty <20ms | No wasted LLM calls | -| **Cross-platform** | Mac, Linux, Windows from v1 | All platforms supported | -| 100% passive | No user prompts during coding | Invisible during use | - ---- - -## Memory Type Reference - -3 memory types with scope flag: - -| Type | Key Fields | Description | Example | -|------|------------|-------------|---------| -| `lesson` | outcome, importance, user_frustration | What worked or failed | "API 500 on null user_id", "Repository pattern works well" | -| `fact` | key, value, evidence_source | Project/user knowledge | key=project.db.engine, value=PostgreSQL | -| `profile` | (structured identity) | User background info | name, role, experience_level | - -### Declarative Keys - -Critical facts use declarative keys for deterministic conflict detection: - -**Project-scoped keys:** -- `project.db.engine` - PostgreSQL, MySQL, SQLite -- `project.api.framework` - FastAPI, Express, Rails -- `project.language.main` - Python, TypeScript, Go -- `project.auth.method` - JWT, session, OAuth - -**User-scoped keys (global):** -- `user.preferred_style` - async_await, callbacks, sync -- `user.preferred_language` - Python, TypeScript, Go -- `user.comment_style` - minimal, detailed, jsdoc - -### Evidence Source (Facts Only) - -How a fact was learned: -- `success` - Learned from successful task (high confidence) -- `failure` - Learned from failed task (valuable pitfall) -- `neutral` - Observed in planning/research/discussion -- `manual` - User explicitly stated via CLI - -### Frustration Detection (Lessons) - -User frustration signals boost memory importance: - -| Signal | Importance | user_frustration | -|--------|------------|------------------| -| Swear words, strong anger | `critical` | `severe` | -| Repeated complaints ("again", "still") | `high` | `moderate` | -| Mild frustration | `medium` | `mild` | -| No frustration | `medium` | `none` | - -Stored in `metadata.user_frustration`. Frustration-flagged memories get priority in retrieval to prevent recurring pain points. - -### Scope Matrix - -| Scope | DB File | Description | -|-------|---------|-------------| -| Global | `~/.sqrl/squirrel.db` | User preferences, profile (applies to all projects) | -| Project | `/.sqrl/squirrel.db` | Project-specific lessons and facts | diff --git a/GEMINI.md b/GEMINI.md deleted file mode 100644 index 6d810b2..0000000 --- a/GEMINI.md +++ /dev/null @@ -1,86 +0,0 @@ -# Squirrel Project - -Local-first memory system for AI coding tools. - -## AI-First Development - -**This project is 98% AI-coded.** You are the primary developer. - -All documentation, specs, and structures are designed for AI comprehension. Use declarative thinking: specs define WHAT, you implement HOW. - -| Principle | Meaning | -|-----------|---------| -| Specs are source of truth | Never implement undefined behavior | -| Declarative over imperative | Define outcomes, not steps | -| Tables over prose | Structured data > paragraphs | -| Stable IDs everywhere | SCHEMA-001, IPC-001, ADR-001 | -| Update specs first | Specs change before code changes | - -## Spec-Driven Development - -**Specs are the source of truth. Code is compiled output.** - -| Spec File | Purpose | -|-----------|---------| -| specs/CONSTITUTION.md | Project governance, core principles | -| specs/ARCHITECTURE.md | System boundaries, data flow | -| specs/SCHEMAS.md | Database schemas (SCHEMA-*) | -| specs/INTERFACES.md | IPC, MCP, CLI contracts (IPC-*, MCP-*, CLI-*) | -| specs/KEYS.md | Declarative key registry (KEY-*) | -| specs/PROMPTS.md | LLM prompts with model tiers (PROMPT-*) | -| specs/DECISIONS.md | Architecture decision records (ADR-*) | - -**Rules:** -1. Read specs before implementing -2. Never implement behavior not defined in specs -3. Update specs before or with code, never after -4. Reference spec IDs in commits (e.g., "implements SCHEMA-001") - -## Project Rules - -See `project-rules/*.mdc` for context-specific rules: -- `general.mdc` - Overall development rules -- `rust-daemon.mdc` - Rust daemon boundaries (ARCH-001) -- `python-agent.mdc` - Python agent boundaries (ARCH-002) -- `specs.mdc` - Specification maintenance -- `testing.mdc` - Testing requirements (DR4) - -## Architecture - -``` -Rust Daemon (I/O, storage, MCP) <--IPC--> Python Agent (LLM operations) -``` - -| Component | Responsibility | Never Does | -|-----------|----------------|------------| -| Rust Daemon | Log watching, MCP server, CLI, SQLite | LLM calls | -| Python Agent | Memory extraction, context composition | File watching | - -## Development Environment - -Uses Nix via devenv (ADR-006). Single command: - -```bash -devenv shell -``` - -## Team Standards - -### Communication -- No unnecessary emojis -- English only in code, comments, commits -- Brief, direct language -- Today's date: 2025 Dec 9 - -### Git Workflow - -Branch: `yourname/type-description` - -Commit: `type(scope): brief description` -- Reference spec IDs when applicable - -### Code Quality -- Write tests for new features (DR4) -- Keep files under 200 lines -- Only change what's necessary (DR5) -- No drive-by refactoring diff --git a/specs/ARCHITECTURE.md b/specs/ARCHITECTURE.md index e79a2cd..9b4f340 100644 --- a/specs/ARCHITECTURE.md +++ b/specs/ARCHITECTURE.md @@ -2,6 +2,28 @@ High-level system boundaries and data flow. +## Technology Stack + +| Category | Technology | Notes | +|----------|------------|-------| +| **Rust Daemon** | | | +| Storage | SQLite + sqlite-vec | Local-first, vector search | +| IPC Protocol | JSON-RPC 2.0 | MCP-compatible, over Unix socket | +| MCP SDK | rmcp | Official Rust SDK (modelcontextprotocol/rust-sdk) | +| CLI Framework | clap | Rust CLI parsing | +| Async Runtime | tokio | Async I/O | +| File Watching | notify | Cross-platform fs events | +| Logging | tracing | Structured logging | +| **Python Agent** | | | +| Agent Framework | PydanticAI | Python agent with tools | +| LLM Client | LiteLLM | Multi-provider support | +| Embeddings | OpenAI text-embedding-3-small | 1536-dim, API-based | +| Logging | structlog | Structured logging | +| **Build & Release** | | | +| Rust Build | cargo-dist | Generates Homebrew, MSI, installers | +| Auto-update | axoupdater | dist's official updater | +| Python Packaging | PyInstaller | Bundled, zero user deps | + ## System Overview ``` diff --git a/specs/CONSTITUTION.md b/specs/CONSTITUTION.md index 459fa66..0b5f46c 100644 --- a/specs/CONSTITUTION.md +++ b/specs/CONSTITUTION.md @@ -75,3 +75,15 @@ Only change what's necessary. No drive-by refactoring. No "while I'm here" impro - No emojis in documentation - Brief, direct language - Tables over paragraphs + +## Agent Instruction Files + +Single source of truth for AI tool instructions: + +| File | Purpose | +|------|---------| +| `AGENTS.md` | Canonical source (Codex native) | +| `.claude/CLAUDE.md` | Symlink → AGENTS.md | +| `.cursor/rules/*.mdc` | Cursor project rules | + +GEMINI.md and .cursorrules are deprecated. Configure Gemini CLI to read AGENTS.md via `contextFileName` setting. diff --git a/specs/CONTRIBUTING.md b/specs/CONTRIBUTING.md new file mode 100644 index 0000000..085cc62 --- /dev/null +++ b/specs/CONTRIBUTING.md @@ -0,0 +1,267 @@ +# Squirrel Contributing Guide + +Collaboration standards for contributors. This document complements CONSTITUTION.md with practical workflow details. + +## Contributor Workflow + +### Branch Strategy + +| Branch | Purpose | Protection | +|--------|---------|------------| +| `main` | Stable release-ready code | Protected, requires PR | +| `/-` | Feature/fix branches | None | + +**Branch naming examples:** +- `adrian/feat-log-watcher` +- `lyrica/fix-ipc-timeout` +- `adrian/docs-contributing` + +### Pull Request Process + +#### PR-001: Standard PR Flow + +``` +1. Create branch from main +2. Make changes (follow spec-driven development) +3. Run local checks: `fmt` + `lint` + `test-all` +4. Push branch +5. Create PR with template +6. CI runs automatically +7. Request review +8. Address feedback +9. Squash merge to main +``` + +#### PR-002: PR Template + +All PRs must include: + +```markdown +## Summary +Brief description of changes (1-2 sentences) + +## Spec References +- Implements: SCHEMA-001, IPC-002 (if applicable) +- Updates: specs/ARCHITECTURE.md (if spec changed) + +## Type +- [ ] feat: New feature +- [ ] fix: Bug fix +- [ ] docs: Documentation only +- [ ] refactor: Code refactoring +- [ ] test: Test additions +- [ ] chore: Build/tooling changes + +## Checklist +- [ ] Specs updated (if behavior changed) +- [ ] Tests added/updated +- [ ] `fmt` passes +- [ ] `lint` passes +- [ ] `test-all` passes + +## Test Plan +How to verify this change works. +``` + +#### PR-003: Review Requirements + +| Change Type | Required Reviewers | Auto-merge | +|-------------|-------------------|------------| +| docs only | 1 | Yes (after CI) | +| code (non-breaking) | 1 | No | +| spec changes | 2 | No | +| breaking changes | 2 + explicit approval | No | + +### Commit Standards + +#### COMMIT-001: Message Format + +``` +(): + +[optional body] + +[optional footer: refs #issue, implements SPEC-ID] +``` + +**Types:** +| Type | Description | +|------|-------------| +| feat | New feature | +| fix | Bug fix | +| docs | Documentation | +| refactor | Code refactoring (no behavior change) | +| test | Test additions/fixes | +| chore | Build, CI, tooling | +| perf | Performance improvement | + +**Scopes:** +| Scope | Description | +|-------|-------------| +| daemon | Rust daemon code | +| agent | Python agent code | +| cli | CLI interface | +| mcp | MCP server | +| ipc | IPC protocol | +| specs | Specification files | +| ci | CI/CD configuration | + +**Examples:** +``` +feat(daemon): implement Claude Code log watcher + +Implements FLOW-001 passive ingestion for Claude Code JSONL files. +Uses notify crate for cross-platform file watching. + +Implements: ARCH-001 +Refs: #12 +``` + +``` +fix(agent): handle empty episode gracefully + +Return early with empty result instead of raising exception +when episode has no events. + +Fixes: #34 +``` + +#### COMMIT-002: Commit Hygiene + +| Rule | Description | +|------|-------------| +| Atomic commits | One logical change per commit | +| Passing state | Each commit should pass CI | +| No WIP commits | Squash before PR, no "WIP", "fixup" | +| Reference specs | Include spec IDs when implementing specs | + +### Code Review Guidelines + +#### REVIEW-001: Reviewer Checklist + +| Check | Question | +|-------|----------| +| Spec alignment | Does code match spec? If no spec, should there be one? | +| Boundary respect | Rust doing I/O only? Python doing LLM only? | +| Error handling | Graceful degradation? No panics in daemon? | +| Tests | New behavior covered? Edge cases? | +| Security | No secrets? Input validated? | +| Performance | Acceptable latency? No unnecessary allocations? | + +#### REVIEW-002: Review Etiquette + +| Do | Don't | +|----|-------| +| Be specific and actionable | Vague criticism | +| Suggest alternatives | Just say "this is wrong" | +| Approve when ready | Block on nitpicks | +| Use "nit:" prefix for optional | Demand perfection | + +### Issue Management + +#### ISSUE-001: Issue Labels + +| Label | Description | Color | +|-------|-------------|-------| +| `bug` | Something broken | Red | +| `feat` | Feature request | Green | +| `docs` | Documentation | Blue | +| `good-first-issue` | Beginner friendly | Purple | +| `help-wanted` | Needs contributor | Yellow | +| `blocked` | Waiting on something | Orange | +| `wontfix` | Not planned | Gray | + +#### ISSUE-002: Issue Template + +```markdown +## Description +Clear description of the issue or feature request. + +## Current Behavior (for bugs) +What happens now. + +## Expected Behavior +What should happen. + +## Reproduction Steps (for bugs) +1. Step one +2. Step two + +## Environment +- OS: +- Squirrel version: +- CLI being used: + +## Spec Reference (if applicable) +Related spec IDs: SCHEMA-001, IPC-002 +``` + +## Security + +### SEC-001: Secrets Handling + +| Rule | Enforcement | +|------|-------------| +| No secrets in code | Pre-commit hook check | +| No secrets in commits | CI secret scanning | +| API keys via env vars | Documented in README | +| `.env` files gitignored | In .gitignore | + +**Forbidden patterns in commits:** +- API keys (sk-*, anthropic-*, etc.) +- Private keys +- Passwords +- Connection strings with credentials + +### SEC-002: Dependency Security + +| Check | Frequency | Tool | +|-------|-----------|------| +| Rust advisories | Every CI run | `cargo audit` | +| Python vulnerabilities | Every CI run | `pip-audit` | +| Dependabot alerts | Automated | GitHub Dependabot | + +## Release Process + +### RELEASE-001: Versioning + +Follow [Semantic Versioning](https://semver.org/): + +``` +MAJOR.MINOR.PATCH + +MAJOR: Breaking changes +MINOR: New features (backward compatible) +PATCH: Bug fixes (backward compatible) +``` + +### RELEASE-002: Release Checklist + +``` +1. [ ] All CI passing on main +2. [ ] CHANGELOG.md updated +3. [ ] Version bumped in Cargo.toml and pyproject.toml +4. [ ] Tag created: v{version} +5. [ ] GitHub Release created with notes +6. [ ] Binaries built and attached (via CI) +7. [ ] Homebrew formula updated (if applicable) +``` + +## Communication + +### COMM-001: Channels + +| Channel | Purpose | +|---------|---------| +| GitHub Issues | Bugs, feature requests | +| GitHub Discussions | Questions, ideas, RFC | +| Pull Requests | Code review | + +### COMM-002: Response Times + +| Type | Target Response | +|------|-----------------| +| Security issues | 24 hours | +| Bug reports | 48 hours | +| Feature requests | 1 week | +| PR reviews | 48 hours | diff --git a/specs/DECISIONS.md b/specs/DECISIONS.md index 1f89043..4591c22 100644 --- a/specs/DECISIONS.md +++ b/specs/DECISIONS.md @@ -241,3 +241,44 @@ Use Unix socket at `/tmp/sqrl_agent.sock` with JSON-RPC 2.0 protocol. Windows us | Team sync backend | Supabase / Custom / None | v2 | | Local LLM support | Ollama / llama.cpp / None | v2 | | Web UI | None / Tauri / Electron | v2 | + +--- + +## Future: v2 Team/Cloud Architecture + +Reference architecture for team memory sharing (not in v1 scope). + +### 3-Layer Database Architecture + +| Layer | DB File | Contents | Sync | +|-------|---------|----------|------| +| Global | `~/.sqrl/squirrel.db` | lesson, fact, profile (scope=global) | Local only | +| Project | `/.sqrl/squirrel.db` | lesson, fact (scope=project) | Local only | +| Team | `~/.sqrl/group.db` | Shared memories (owner=team) | Cloud | + +### Memory Schema Extensions (v2) + +```sql +-- Additional fields for team support +ALTER TABLE memories ADD COLUMN owner TEXT NOT NULL DEFAULT 'individual'; -- individual | team +ALTER TABLE memories ADD COLUMN team_id TEXT; -- team identifier +ALTER TABLE memories ADD COLUMN contributed_by TEXT; -- user who shared +ALTER TABLE memories ADD COLUMN source_memory_id TEXT; -- original memory ID +``` + +### Scaling Strategy + +| Team Size | Strategy | +|-----------|----------| +| Small (<100) | Full sync - all team memories in local group.db | +| Medium (100-1000) | Partial sync - recent + relevant memories locally | +| Large (1000+) | Cloud-primary - query cloud, cache locally | + +### Team Commands (v2) + +```bash +sqrl team join # Join team, start syncing group.db +sqrl team leave # Leave team, remove group.db +sqrl share # Promote individual memory to team +sqrl team export # Export team memories to local +```