diff --git a/.claude/commands/check-db.md b/.claude/commands/check-db.md new file mode 100644 index 0000000..2da92e2 --- /dev/null +++ b/.claude/commands/check-db.md @@ -0,0 +1,29 @@ +--- +description: Verify RootsMagic database file exists and is accessible +--- + +Check that the RootsMagic database file is present and can be queried. + +```bash +DB_PATH="${RM_DATABASE_PATH:-data/Iiams.rmtree}" + +if [ ! -f "$DB_PATH" ]; then + echo "❌ Database file not found: $DB_PATH" + echo "" + echo "Set RM_DATABASE_PATH in config/.env" + exit 1 +fi + +echo "✅ Database found: $DB_PATH" +echo "" + +# Get basic stats +PERSON_COUNT=$(sqlite3 "$DB_PATH" "SELECT COUNT(*) FROM PersonTable;") +EVENT_COUNT=$(sqlite3 "$DB_PATH" "SELECT COUNT(*) FROM EventTable;") +SOURCE_COUNT=$(sqlite3 "$DB_PATH" "SELECT COUNT(*) FROM SourceTable;") + +echo "📊 Database Statistics:" +echo " • Persons: $PERSON_COUNT" +echo " • Events: $EVENT_COUNT" +echo " • Sources: $SOURCE_COUNT" +``` diff --git a/.claude/commands/coverage.md b/.claude/commands/coverage.md new file mode 100644 index 0000000..1dd5556 --- /dev/null +++ b/.claude/commands/coverage.md @@ -0,0 +1,11 @@ +--- +description: Run tests with coverage report +--- + +Run the full test suite with coverage analysis. Shows which lines are covered and identifies gaps. + +```bash +uv run pytest --cov=rmagent --cov-report=term-missing --cov-report=html +echo "" +echo "📊 Coverage report generated at: file://$(pwd)/htmlcov/index.html" +``` diff --git a/.claude/commands/doc-review.md b/.claude/commands/doc-review.md new file mode 100644 index 0000000..cdac1df --- /dev/null +++ b/.claude/commands/doc-review.md @@ -0,0 +1,149 @@ +--- +description: Review documentation for accuracy and completeness +show-prompt: true +meta: true +--- + +Review project documentation to ensure it's concise, current, and accurate. + +**Modes:** +- `/doc-review` or `/doc-review brief` - Review root docs and INDEX.md +- `/doc-review deep` - Review ALL documentation files + +This command reads documentation files and asks the LLM to verify: +1. Content is concise and well-organized +2. Information is current (no outdated references) +3. Cross-references and links are accurate +4. No contradictions between documents +5. INDEX.md accurately reflects documentation structure + +```bash +MODE="${1:-brief}" + +if [ "$MODE" = "deep" ]; then + cat <<'PROMPT' +Please perform a DEEP documentation review of the RMAgent project. + +Review ALL documentation files for: +1. **Accuracy** - Are all statements, commands, and examples correct? +2. **Currency** - Is information up-to-date? Any outdated references? +3. **Consistency** - Do documents contradict each other? +4. **Completeness** - Are there gaps in documentation coverage? +5. **Conciseness** - Can any content be condensed without losing value? +6. **Organization** - Is content in the right location? + +**Root Documentation Files:** +PROMPT + + echo "" + echo "=== CLAUDE.md ===" + head -100 CLAUDE.md + echo "" + echo "=== README.md ===" + head -100 README.md + echo "" + echo "=== AGENTS.md ===" + head -50 AGENTS.md + echo "" + echo "=== CONTRIBUTING.md ===" + head -50 CONTRIBUTING.md + echo "" + echo "=== CHANGELOG.md ===" + head -30 CHANGELOG.md + + cat <<'PROMPT' + +**Documentation Index:** +PROMPT + + echo "" + echo "=== docs/INDEX.md ===" + cat docs/INDEX.md + + cat <<'PROMPT' + +**All Documentation Files:** +PROMPT + + echo "" + find docs -name "*.md" -type f | sort | while read file; do + echo "" + echo "=== $file ===" + head -80 "$file" + echo "... [file continues]" + done + + cat <<'PROMPT' + +Please provide: +1. **Overall Assessment** - General state of documentation +2. **Critical Issues** - Must-fix problems (inaccuracies, broken links, outdated info) +3. **Recommendations** - Suggestions for improvement +4. **Specific Fixes** - Line-by-line corrections needed + +Focus on actionable feedback that improves documentation quality. +PROMPT + +else + # Brief mode - just check root docs + INDEX.md + cat <<'PROMPT' +Please perform a BRIEF documentation review of the RMAgent project's core files. + +Review the following files for: +1. **Accuracy** - Are statements, commands, and statistics correct? +2. **Currency** - Any outdated references or old information? +3. **Consistency** - Do these files contradict each other? +4. **Conciseness** - Is content appropriately detailed (not too verbose)? +5. **INDEX.md Accuracy** - Does INDEX.md correctly reference all docs in the repository? + +**Root Documentation Files:** +PROMPT + + echo "" + echo "=== CLAUDE.md ===" + cat CLAUDE.md + echo "" + echo "=== README.md ===" + cat README.md + echo "" + echo "=== AGENTS.md ===" + cat AGENTS.md + echo "" + echo "=== CONTRIBUTING.md ===" + cat CONTRIBUTING.md + echo "" + echo "=== CHANGELOG.md ===" + cat CHANGELOG.md + + cat <<'PROMPT' + +**Documentation Index:** +PROMPT + + echo "" + echo "=== docs/INDEX.md ===" + cat docs/INDEX.md + + cat <<'PROMPT' + +**Verify INDEX.md Completeness:** +Check that docs/INDEX.md accurately references all documentation files in the repository. +PROMPT + + echo "" + echo "=== All docs files in repository ===" + find docs -name "*.md" -type f | sort + + cat <<'PROMPT' + +Please provide: +1. **Quick Assessment** - Overall state of core documentation +2. **Critical Issues** - Any inaccuracies, broken references, or outdated info +3. **INDEX.md Status** - Is it complete and accurate? +4. **Quick Wins** - Easy improvements to make immediately + +Be concise and actionable. Focus on what needs fixing right now. +PROMPT + +fi +``` diff --git a/.claude/commands/docs.md b/.claude/commands/docs.md new file mode 100644 index 0000000..420d3c9 --- /dev/null +++ b/.claude/commands/docs.md @@ -0,0 +1,33 @@ +--- +description: Open documentation in browser or show quick reference +--- + +Quick access to RMAgent documentation. + +Usage: +- `/docs` - Show documentation index +- `/docs schema` - Open schema reference +- `/docs data-formats` - Open data formats reference +- `/docs dev` - Open developer guide + +```bash +case "$ARGUMENTS" in + schema) + echo "📖 Schema Reference: docs/reference/schema/schema-reference.md" + cat docs/reference/schema/schema-reference.md | head -50 + ;; + data-formats) + echo "📖 Data Formats: docs/reference/data-formats/" + ls -1 docs/reference/data-formats/ + ;; + dev) + echo "📖 Developer Guide: docs/guides/developer-guide.md" + cat docs/guides/developer-guide.md | head -50 + ;; + *) + echo "📚 RMAgent Documentation Index" + echo "" + cat docs/INDEX.md | head -80 + ;; +esac +``` diff --git a/.claude/commands/lint.md b/.claude/commands/lint.md new file mode 100644 index 0000000..d13d387 --- /dev/null +++ b/.claude/commands/lint.md @@ -0,0 +1,22 @@ +--- +description: Run linting checks (ruff and black) +--- + +Run code quality checks using ruff and black formatters. + +Usage: +- `/lint` - Check code without modifying +- `/lint fix` - Auto-fix issues where possible + +```bash +if [ "$ARGUMENTS" = "fix" ]; then + echo "🔧 Auto-fixing issues..." + uv run ruff check --fix . + uv run black . + echo "✅ Fixes applied" +else + echo "🔍 Checking code quality..." + uv run ruff check . + uv run black --check . +fi +``` diff --git a/.claude/commands/rm-ask.md b/.claude/commands/rm-ask.md new file mode 100644 index 0000000..e628a91 --- /dev/null +++ b/.claude/commands/rm-ask.md @@ -0,0 +1,23 @@ +--- +description: Ask a question about the genealogy database using AI +show-prompt: true +meta: true +--- + +Ask natural language questions about your genealogy data. Uses AI to query and analyze the database. + +Usage: +- `/rm-ask "Who are the ancestors of John Smith?"` +- `/rm-ask "Find everyone born in Baltimore"` +- `/rm-ask "Show family relationships for person 1"` + +```bash +if [ -z "$ARGUMENTS" ]; then + echo "❌ Error: Question required" + echo "Usage: /rm-ask \"Your question here\"" + exit 1 +fi + +echo "🤔 Asking AI about your genealogy data..." +uv run rmagent ask "$ARGUMENTS" +``` diff --git a/.claude/commands/rm-bio.md b/.claude/commands/rm-bio.md new file mode 100644 index 0000000..3d1a3be --- /dev/null +++ b/.claude/commands/rm-bio.md @@ -0,0 +1,27 @@ +--- +description: Generate a biography for a person using rmagent +show-prompt: true +meta: true +--- + +Generate a biography using the rmagent CLI. Requires a person ID. + +Usage: +- `/rm-bio 1` - Generate standard biography for person 1 +- `/rm-bio 1 short` - Generate short biography +- `/rm-bio 1 comprehensive` - Generate comprehensive biography + +```bash +if [ -z "$1" ]; then + echo "❌ Error: Person ID required" + echo "Usage: /rm-bio [length]" + echo "Example: /rm-bio 1 standard" + exit 1 +fi + +PERSON_ID=$1 +LENGTH=${2:-standard} + +echo "📝 Generating $LENGTH biography for person $PERSON_ID..." +uv run rmagent bio $PERSON_ID --length $LENGTH --output reports/biographies/ +``` diff --git a/.claude/commands/rm-person.md b/.claude/commands/rm-person.md new file mode 100644 index 0000000..a980dc5 --- /dev/null +++ b/.claude/commands/rm-person.md @@ -0,0 +1,23 @@ +--- +description: Query person details from RootsMagic database +--- + +Query detailed information about a person using rmagent CLI. + +Usage: +- `/rm-person 1` - Basic person info +- `/rm-person 1 --events` - Include all events +- `/rm-person 1 --family` - Include family relationships +- `/rm-person 1 --ancestors` - Include ancestors +- `/rm-person 1 --descendants` - Include descendants + +```bash +if [ -z "$1" ]; then + echo "❌ Error: Person ID required" + echo "Usage: /rm-person [--options]" + exit 1 +fi + +echo "👤 Querying person $1..." +uv run rmagent person $@ +``` diff --git a/.claude/commands/rm-quality.md b/.claude/commands/rm-quality.md new file mode 100644 index 0000000..d47b57f --- /dev/null +++ b/.claude/commands/rm-quality.md @@ -0,0 +1,22 @@ +--- +description: Run data quality validation on RootsMagic database +--- + +Run data quality checks on the RootsMagic database. + +Usage: +- `/rm-quality` - Run all quality checks +- `/rm-quality dates` - Check only date issues +- `/rm-quality names` - Check only name issues +- `/rm-quality places` - Check only place issues +- `/rm-quality relationships` - Check only relationship issues + +```bash +if [ -z "$ARGUMENTS" ]; then + echo "🔍 Running all data quality checks..." + uv run rmagent quality --format table +else + echo "🔍 Running quality checks for category: $ARGUMENTS" + uv run rmagent quality --category $ARGUMENTS --format table +fi +``` diff --git a/.claude/commands/rm-search.md b/.claude/commands/rm-search.md new file mode 100644 index 0000000..6d1681b --- /dev/null +++ b/.claude/commands/rm-search.md @@ -0,0 +1,21 @@ +--- +description: Search RootsMagic database by name or place +--- + +Search the RootsMagic database for people by name or place. + +Usage: +- `/rm-search --name "John Smith"` - Search by name +- `/rm-search --place "Baltimore"` - Search by place +- `/rm-search --name "Smith" --limit 10` - Limit results + +```bash +if [ -z "$ARGUMENTS" ]; then + echo "❌ Error: Search parameters required" + echo "Usage: /rm-search --name \"Name\" or /rm-search --place \"Place\"" + exit 1 +fi + +echo "🔎 Searching database..." +uv run rmagent search $ARGUMENTS +``` diff --git a/.claude/commands/rm-timeline.md b/.claude/commands/rm-timeline.md new file mode 100644 index 0000000..9600c66 --- /dev/null +++ b/.claude/commands/rm-timeline.md @@ -0,0 +1,22 @@ +--- +description: Generate timeline visualization for a person +--- + +Generate a TimelineJS3 timeline for a person's life events. + +Usage: +- `/rm-timeline 1` - Generate JSON timeline for person 1 +- `/rm-timeline 1 --format html` - Generate HTML timeline +- `/rm-timeline 1 --include-family` - Include family events +- `/rm-timeline 1 --group-by-phase` - Group events by life phases + +```bash +if [ -z "$1" ]; then + echo "❌ Error: Person ID required" + echo "Usage: /rm-timeline [options]" + exit 1 +fi + +echo "📅 Generating timeline for person $1..." +uv run rmagent timeline $@ +``` diff --git a/.claude/commands/test.md b/.claude/commands/test.md new file mode 100644 index 0000000..a6088f5 --- /dev/null +++ b/.claude/commands/test.md @@ -0,0 +1,19 @@ +--- +description: Run pytest tests with optional filters +--- + +Run the test suite using pytest. You can optionally specify a test file or pattern. + +Usage: +- `/test` - Run all tests +- `/test unit` - Run only unit tests +- `/test integration` - Run only integration tests +- `/test test_queries.py` - Run specific test file + +```bash +if [ -z "$ARGUMENTS" ]; then + uv run pytest -v +else + uv run pytest -v tests/$ARGUMENTS* +fi +``` diff --git a/CLAUDE.md b/CLAUDE.md index 3261928..c54266a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,40 +24,45 @@ rmagent/ │ └── rmlib/ # Core library (database, parsers, queries) ├── config/ # Runtime config (config/.env) ├── data/ # Database files (*.rmtree, NOT tracked in git) -├── data_reference/ # 18 schema/format docs (RM11_*.md) -├── docs/ # Project docs (AI_AGENT_TODO.md, USER_GUIDE.md, MVP_CHECKPOINT.md) +├── docs/ # **📚 START HERE: docs/INDEX.md** - Complete documentation +│ ├── INDEX.md # Master table of contents +│ ├── getting-started/ # Installation, quickstart, configuration +│ ├── guides/ # User & developer guides +│ ├── reference/ # Schema, formats, query patterns +│ ├── projects/ # Active feature development +│ └── archive/ # Completed milestones & summaries ├── sqlite-extension/ # ICU extension for RMNOCASE collation -└── tests/unit/ # Test suite (245+ tests, pytest) +└── tests/unit/ # Test suite (490+ tests, pytest) ``` -## Essential Documentation (data_reference/) +## 📚 Essential Documentation -**Schema & Structure:** -- **RM11_Schema_Reference.md** - START HERE: tables, fields, relationships, query patterns -- **RM11_schema_annotated.sql** - SQL with comments for query writing -- **RM11_DataDef.yaml** - Field enumerations and constraints +**For complete documentation, see [`docs/INDEX.md`](docs/INDEX.md)** -**Core Formats:** -- **RM11_Date_Format.md** - CRITICAL: 24-char date encoding (ranges, qualifiers, BC/AD) -- **RM11_Place_Format.md** - Comma-delimited hierarchy (City, County, State, Country) -- **RM11_FactTypes.md** - 65 built-in event types +### Quick Reference (Most Important Files) -**BLOB Structures (UTF-8 XML with BOM):** -- **RM11_BLOB_SourceFields.md** - SourceTable.Fields extraction -- **RM11_BLOB_SourceTemplateFieldDefs.md** - Template definitions (433 templates) -- **RM11_BLOB_CitationFields.md** - CitationTable.Fields extraction +**Schema & Database:** +- **[schema-reference.md](docs/reference/schema/schema-reference.md)** - START HERE: tables, fields, relationships +- **[annotated-schema.sql](docs/reference/schema/annotated-schema.sql)** - SQL with comments +- **[data-definitions.yaml](docs/reference/schema/data-definitions.yaml)** - Field enumerations -**Data Quality & Output:** -- **RM11_Data_Quality_Rules.md** - 24 validation rules across 6 categories -- **RM11_Query_Patterns.md** - 15 optimized SQL patterns -- **RM11_Biography_Best_Practices.md** - 9-section structure, citation styles -- **RM11_Timeline_Construction.md** - TimelineJS3 JSON generation +**Critical Data Formats:** +- **[date-format.md](docs/reference/data-formats/date-format.md)** - ⚠️ CRITICAL: 24-char date encoding +- **[place-format.md](docs/reference/data-formats/place-format.md)** - Comma-delimited hierarchy +- **[fact-types.md](docs/reference/data-formats/fact-types.md)** - 65 built-in event types -**Additional References:** -- RM11_Relationships.md (Relate1/Relate2 calculations) -- RM11_Name_Display_Logic.md (context-aware name selection) -- RM11_EventTable_Details.md (Details field patterns) -- RM11_Sentence_Templates.md (reference only - AI generates text natively) +**BLOB Parsing (UTF-8 XML with BOM):** +- **[blob-source-fields.md](docs/reference/data-formats/blob-source-fields.md)** - SourceTable.Fields +- **[blob-citation-fields.md](docs/reference/data-formats/blob-citation-fields.md)** - CitationTable.Fields +- **[blob-template-field-defs.md](docs/reference/data-formats/blob-template-field-defs.md)** - Template definitions + +**Query & Quality:** +- **[query-patterns.md](docs/reference/query-patterns/query-patterns.md)** - 15 optimized SQL patterns +- **[data-quality-rules.md](docs/reference/query-patterns/data-quality-rules.md)** - 24 validation rules + +**Biography & Output:** +- **[biography-best-practices.md](docs/reference/biography/biography-best-practices.md)** - 9-section structure +- **[timeline-construction.md](docs/reference/biography/timeline-construction.md)** - TimelineJS3 format ## Critical Schema Patterns @@ -106,6 +111,26 @@ RM_DATABASE_PATH=data/Iiams.rmtree LOG_LEVEL=DEBUG # Enable LLM logging ``` +### Claude Code Integration + +RMAgent includes 12 custom slash commands and automated hooks for Claude Code: + +**Quick Commands:** +- `/rm-bio ` - Generate biography with AI +- `/rm-person ` - Query person from database +- `/rm-quality` - Run data quality checks +- `/doc-review [brief|deep]` - Review documentation for accuracy with AI +- `/test` - Run pytest suite +- `/coverage` - Run tests with coverage +- `/check-db` - Verify database connection + +**Automated Hooks:** +- Coverage reminders after pytest runs +- Commit preview before git push +- Documentation review reminder (pre-commit) + +**See [`docs/guides/claude-code-setup.md`](docs/guides/claude-code-setup.md) for complete setup and usage guide.** + ## Project Status (2025-10-12) 🎉 **Milestone 2: MVP ACHIEVED** - All foundation phases complete (33/33 tasks) @@ -129,11 +154,11 @@ LOG_LEVEL=DEBUG # Enable LLM logging - Source formatting improvements (italic rendering, type prefix removal) - Biography collision handling with sequential numbering -**Test Coverage:** 418 tests, 82% overall coverage (97% database, 96% parsers, 91% quality) +**Test Coverage:** 490 tests, 88% overall coverage (97% database, 96% parsers, 94% rendering) **Next Phase:** Phase 7 - Production Polish (performance optimization, advanced features) -See `docs/AI_AGENT_TODO.md` for complete roadmap. +See [`docs/projects/ai-agent/roadmap.md`](docs/projects/ai-agent/roadmap.md) for complete roadmap. ## CLI Commands @@ -159,7 +184,7 @@ All commands use `uv run rmagent [command]`: ## LangChain v1.0 Integration (Future) -**Status:** Zero active LangChain imports. v1.0 upgrade planned for Phase 7. See `docs/RM11_LangChain_Upgrade.md` and `AGENTS.md` for patterns. +**Status:** Zero active LangChain imports. v1.0 upgrade planned for Phase 7. See [`docs/projects/ai-agent/langchain-upgrade.md`](docs/projects/ai-agent/langchain-upgrade.md) for detailed plan. **v1.0 Requirements:** `create_agent()`, `system_prompt="string"`, TypedDict state only. New code goes in `rmagent/agent/lc/` directory. @@ -205,7 +230,7 @@ All PRs automatically run: - Full test suite with coverage (must maintain 80%+) - See `.github/workflows/pr-tests.yml` -**For detailed workflow instructions, see `docs/GIT_WORKFLOW_GUIDE.md`** +**For detailed workflow instructions, see [`docs/guides/git-workflow.md`](docs/guides/git-workflow.md)** ## Quick Reference diff --git a/README.md b/README.md index 267c868..89883ec 100644 --- a/README.md +++ b/README.md @@ -132,7 +132,7 @@ quality_summary = agent.analyze_data_quality() ### CLI Setup Options **Option 1: Direct Access (Recommended)** -Run `./setup_cli.sh` to enable direct CLI access and tab completion. See [docs/CLI_SETUP.md](docs/CLI_SETUP.md) for details. +Run `./setup_cli.sh` to enable direct CLI access and tab completion. After setup, use commands directly: ```bash @@ -377,13 +377,12 @@ When LangChain v1.0 stable releases, use these patterns: ### Migration Plan -See `docs/RM11_LangChain_Upgrade.md` for complete upgrade strategy and timeline. +See [`docs/projects/ai-agent/langchain-upgrade.md`](docs/projects/ai-agent/langchain-upgrade.md) for complete upgrade strategy and timeline. **Key Points:** - New LangChain code goes in `rmagent/agent/lc/` directory - Use v1.0 patterns from day one (no migration needed) - Maintain 80%+ test coverage for all LangChain features -- See `AGENTS.md` for comprehensive best practices ## Development @@ -414,33 +413,34 @@ uv run pytest --cov=rmagent --cov-report=html ## Documentation -**📚 Complete Documentation Index:** [docs/README.md](docs/README.md) +**📚 Complete Documentation Index:** **[docs/INDEX.md](docs/INDEX.md)** ← START HERE ### For New Users -Start here to get up and running: +Get up and running quickly: -1. **[INSTALL.md](INSTALL.md)** - Installation guide (macOS, Linux, Windows/WSL2) -2. **[CONFIGURATION.md](CONFIGURATION.md)** - Configuration, LLM providers, prompt customization -3. **[USAGE.md](USAGE.md)** - Complete CLI reference with 50+ examples -4. **[FAQ.md](FAQ.md)** - Common questions and troubleshooting - -**Comprehensive Guide:** [docs/USER_GUIDE.md](docs/USER_GUIDE.md) (31KB, all-in-one) +1. **[Installation Guide](docs/getting-started/installation.md)** - Install RMAgent and dependencies +2. **[Quick Start](docs/getting-started/quickstart.md)** - 5-minute tutorial +3. **[Configuration Guide](docs/getting-started/configuration.md)** - Set up API keys and database +4. **[User Guide](docs/guides/user-guide.md)** - Complete CLI reference with examples +5. **[FAQ](docs/faq.md)** - Troubleshooting and common questions ### For Developers -Start here to contribute or extend RMAgent: +Contribute or extend RMAgent: -1. **[DEVELOPER_GUIDE.md](DEVELOPER_GUIDE.md)** - Architecture, design patterns, API reference +1. **[Developer Guide](docs/guides/developer-guide.md)** - Architecture, design patterns, API reference 2. **[CONTRIBUTING.md](CONTRIBUTING.md)** - Contribution workflow and coding standards -3. **[TESTING.md](TESTING.md)** - Testing guide (279 tests, coverage analysis) -4. **[CHANGELOG.md](CHANGELOG.md)** - Complete version history +3. **[Testing Guide](docs/guides/testing-guide.md)** - Testing guide (490 tests, 88% coverage) +4. **[Git Workflow](docs/guides/git-workflow.md)** - Branching strategy and PR process +5. **[CHANGELOG.md](CHANGELOG.md)** - Version history -### Additional Documentation +### Technical Reference -- **[AGENTS.md](AGENTS.md)** - Agent design patterns -- **[data_reference/](data_reference/)** - RootsMagic 11 schema (18 reference docs) -- **[docs/](docs/)** - Project documentation and completion reports +- **[Schema Reference](docs/reference/schema/)** - RootsMagic 11 database schema +- **[Data Formats](docs/reference/data-formats/)** - Date/place/BLOB formats +- **[Query Patterns](docs/reference/query-patterns/)** - Optimized SQL patterns +- **[Biography Reference](docs/reference/biography/)** - Biography generation guidelines ## Status @@ -450,7 +450,7 @@ Start here to contribute or extend RMAgent: **Completion:** All 26 foundation tasks complete (Phases 1-4) **Next Focus:** Testing & Quality improvements (Phase 5) -See [docs/MVP_CHECKPOINT.md](docs/MVP_CHECKPOINT.md) for complete verification report. +See [docs/archive/checkpoints/mvp-checkpoint.md](docs/archive/checkpoints/mvp-checkpoint.md) for complete verification report. --- @@ -501,9 +501,9 @@ See [docs/MVP_CHECKPOINT.md](docs/MVP_CHECKPOINT.md) for complete verification r - ✅ Export Command (Hugo blog export with batch support, 8 tests, 74% coverage) - ✅ Search Command (name/place search with phonetic matching, 8 tests, 88% coverage) -**⏭️ Next Tasks:** Phase 5 - Testing & Quality (comprehensive integration testing) +**⏭️ Next Tasks:** Phase 7 - Production Polish (performance optimization, advanced features) -See `docs/AI_AGENT_TODO.md` for detailed progress and roadmap. +See [`docs/projects/ai-agent/roadmap.md`](docs/projects/ai-agent/roadmap.md) for detailed progress and roadmap. ## Repository diff --git a/data/Iiams.rmtree-shm b/data/Iiams.rmtree-shm new file mode 100644 index 0000000..fe9ac28 Binary files /dev/null and b/data/Iiams.rmtree-shm differ diff --git a/data/Iiams.rmtree-wal b/data/Iiams.rmtree-wal new file mode 100644 index 0000000..e69de29 diff --git a/docs/INDEX.md b/docs/INDEX.md new file mode 100644 index 0000000..b0a5962 --- /dev/null +++ b/docs/INDEX.md @@ -0,0 +1,172 @@ +# RMAgent Documentation Index + +**Version**: 1.0 +**Last Updated**: 2025-10-15 + +This is the master table of contents for all RMAgent documentation. All paths are relative to the `docs/` directory. + +--- + +## Quick Links + +- [Installation Guide](getting-started/installation.md) - Get RMAgent installed +- [Quick Start](getting-started/quickstart.md) - 5-minute getting started guide +- [User Guide](guides/user-guide.md) - Complete CLI reference +- [Developer Guide](guides/developer-guide.md) - Contributing and development setup +- [FAQ](faq.md) - Frequently asked questions + +--- + +## 📚 Documentation Structure + +### Getting Started + +New users start here: + +- **[installation.md](getting-started/installation.md)** - Installation instructions (uv, dependencies, database setup) +- **[quickstart.md](getting-started/quickstart.md)** - 5-minute tutorial with first commands +- **[configuration.md](getting-started/configuration.md)** - Environment configuration and API keys + +### User Guides + +Complete guides for using RMAgent: + +- **[user-guide.md](guides/user-guide.md)** - Complete CLI command reference and usage patterns +- **[faq.md](faq.md)** - Frequently asked questions and troubleshooting + +### Developer Guides + +For contributors and developers: + +- **[developer-guide.md](guides/developer-guide.md)** - Architecture, coding standards, development setup +- **[testing-guide.md](guides/testing-guide.md)** - Running tests, writing tests, coverage +- **[git-workflow.md](guides/git-workflow.md)** - Branching strategy, commit conventions, PR process +- **[claude-code-setup.md](guides/claude-code-setup.md)** - Claude Code slash commands and hooks configuration + +### Reference Documentation + +Technical reference materials: + +#### Schema & Database +- **[reference/schema/](reference/schema/)** - RootsMagic 11 database schema documentation + - `schema-reference.md` - Complete table and field reference + - `annotated-schema.sql` - Annotated SQL schema + - `data-definitions.yaml` - Field enumerations and constraints + - `relationships.md` - Parent-child and family relationships + +#### Data Formats +- **[reference/data-formats/](reference/data-formats/)** - RootsMagic data format specifications + - `date-format.md` - 24-character date encoding (CRITICAL) + - `place-format.md` - Comma-delimited place hierarchy + - `blob-fields.md` - UTF-8 XML BLOB parsing (sources, citations, templates) + - `fact-types.md` - 65 built-in event types + +#### Query Patterns +- **[reference/query-patterns/](reference/query-patterns/)** - SQL query patterns and best practices + - `query-patterns.md` - 15 optimized query patterns + - `rmnocase-collation.md` - ICU extension for case-insensitive text matching + - `data-quality-rules.md` - 24 validation rules across 6 categories + +#### Biography +- **[reference/biography/](reference/biography/)** - Biography generation reference + - `biography-best-practices.md` - 9-section structure, citation styles + - `timeline-construction.md` - TimelineJS3 JSON generation + +### Active Projects + +Point-in-time feature work and planning documents: + +#### AI Agent Development +- **[projects/ai-agent/](projects/ai-agent/)** + - `roadmap.md` - AI agent development roadmap (phases 1-7) + - `langchain-upgrade.md` - LangChain 1.0 migration plan + - `multi-agent-plan.md` - Multi-agent architecture design + +#### Census Extraction +- **[projects/census-extraction/](projects/census-extraction/)** + - `architecture.md` - Census extraction system architecture + - `implementation-plan.md` - Step-by-step implementation plan + +#### Biography & Citations +- **[projects/biography-citations/](projects/biography-citations/)** + - `citation-implementation.md` - Citation processing implementation + - `married-name-search.md` - Married name search optimization + +### Archive + +Completed milestones and historical documentation: + +- **[archive/checkpoints/](archive/checkpoints/)** - Milestone completion documents + - `mvp-checkpoint.md` - MVP milestone (Phases 1-6 complete) + - `phase-5-completion.md` - Testing & quality phase + - `phase-6-completion.md` - Documentation phase + +- **[archive/summaries/](archive/summaries/)** - Implementation summaries + - `integration-testing-summary.md` - Integration test implementation + - `optimization-summary.md` - Performance optimization work + - `test-coverage-analysis.md` - Coverage improvement analysis + +### Root Documentation + +Key files in the repository root (not in docs/): + +- **[CLAUDE.md](../CLAUDE.md)** - AI assistant context and project guide +- **[AGENTS.md](../AGENTS.md)** - LangChain patterns and multi-agent architecture +- **[README.md](../README.md)** - Repository entry point and quick start +- **[CONTRIBUTING.md](../CONTRIBUTING.md)** - Contribution guidelines +- **[CHANGELOG.md](../CHANGELOG.md)** - Version history and release notes + +--- + +## 🎯 For Claude Code and AI Assistants + +When working on this codebase: + +1. **Start with**: `CLAUDE.md` (root) for project context and guidelines +2. **Schema questions**: `reference/schema/schema-reference.md` +3. **Data parsing**: `reference/data-formats/` (especially `date-format.md`) +4. **Query patterns**: `reference/query-patterns/query-patterns.md` +5. **Active work**: `projects/` subdirectories for current feature development +6. **Testing**: `guides/testing-guide.md` + +--- + +## 📝 Documentation Maintenance + +### Keeping Docs Current + +- **Active work** → `projects/` (move to `archive/` when complete) +- **Completed milestones** → `archive/checkpoints/` +- **Implementation notes** → `archive/summaries/` +- **Update INDEX.md** when adding/moving documentation + +### Naming Conventions + +- **Directories**: `lowercase-with-hyphens/` +- **Files**: `lowercase-with-hyphens.md` +- **Be descriptive**: `installation.md` not `install.md` +- **No abbreviations** unless universally known (e.g., `faq.md` is OK) + +### Where Does It Go? + +| Content Type | Location | Example | +|--------------|----------|---------| +| User-facing guides | `guides/` | CLI reference, tutorials | +| Installation/setup | `getting-started/` | Installation, configuration | +| Technical reference | `reference/` | Schema, formats, patterns | +| Active feature work | `projects/{feature}/` | Planning, architecture docs | +| Completed work | `archive/` | Checkpoints, summaries | +| General questions | `faq.md` | Common issues, how-tos | + +--- + +## 🔗 External Resources + +- **Repository**: https://github.com/miams/rmagent +- **RootsMagic**: https://www.rootsmagic.com/ +- **SQLite**: https://www.sqlite.org/docs.html +- **LangChain**: https://python.langchain.com/ + +--- + +**Need help?** Check [faq.md](faq.md) or open an issue on GitHub. diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 4997f9e..0000000 --- a/docs/README.md +++ /dev/null @@ -1,224 +0,0 @@ -# RMAgent Documentation Index - -Comprehensive documentation for RMAgent - AI-powered genealogy assistant for RootsMagic 11 databases. - -## 📚 Documentation Overview - -### For New Users - -Start here if you're new to RMAgent: - -1. **[README.md](../README.md)** - Project overview and quick start -2. **[INSTALL.md](../INSTALL.md)** - Installation guide (all platforms) -3. **[CONFIGURATION.md](../CONFIGURATION.md)** - Configuration and setup -4. **[USAGE.md](../USAGE.md)** - Complete CLI command reference -5. **[FAQ.md](../FAQ.md)** - Common questions and troubleshooting - -### For Developers - -Start here if you want to contribute or extend RMAgent: - -1. **[DEVELOPER_GUIDE.md](../DEVELOPER_GUIDE.md)** - Architecture, design patterns, API reference -2. **[CONTRIBUTING.md](../CONTRIBUTING.md)** - Contribution workflow -3. **[TESTING.md](../TESTING.md)** - Testing guide -4. **[CHANGELOG.md](../CHANGELOG.md)** - Version history - -### Additional Documentation - -- **[USER_GUIDE.md](USER_GUIDE.md)** - Comprehensive user guide (31KB, all-in-one) -- **[AGENTS.md](../AGENTS.md)** - Agent design patterns - ---- - -## 📖 Documentation by Topic - -### Installation & Setup - -| Document | Description | Audience | -|----------|-------------|----------| -| [INSTALL.md](../INSTALL.md) | Platform-specific installation (macOS, Linux, Windows/WSL2) | All users | -| [CONFIGURATION.md](../CONFIGURATION.md) | Environment variables, LLM providers, prompt customization | All users | -| [FAQ.md](../FAQ.md) | Troubleshooting installation issues | All users | - -### Using RMAgent - -| Document | Description | Audience | -|----------|-------------|----------| -| [USAGE.md](../USAGE.md) | All 7 CLI commands with 50+ examples | All users | -| [USER_GUIDE.md](USER_GUIDE.md) | Comprehensive guide (installation → advanced usage) | All users | -| [FAQ.md](../FAQ.md) | Common workflows and troubleshooting | All users | - -### Development - -| Document | Description | Audience | -|----------|-------------|----------| -| [DEVELOPER_GUIDE.md](../DEVELOPER_GUIDE.md) | Architecture, API reference, adding features | Developers | -| [CONTRIBUTING.md](../CONTRIBUTING.md) | Git workflow, coding standards, PR process | Contributors | -| [TESTING.md](../TESTING.md) | Test suite guide (279 tests, coverage analysis) | Developers | -| [CHANGELOG.md](../CHANGELOG.md) | Complete version history | All | - -### Technical Reference - -| Document | Description | Audience | -|----------|-------------|----------| -| [AGENTS.md](../AGENTS.md) | Agent design patterns | Developers | -| [data_reference/](../data_reference/) | RootsMagic 11 schema (18 docs) | Developers | - ---- - -## 🎯 Quick Navigation - -### By Task - -**I want to...** - -- **Install RMAgent** → [INSTALL.md](../INSTALL.md) -- **Configure API keys** → [CONFIGURATION.md](../CONFIGURATION.md) (LLM Provider Setup) -- **Generate a biography** → [USAGE.md](../USAGE.md) (Biography Command) -- **Customize prompts** → [CONFIGURATION.md](../CONFIGURATION.md) (Prompt Customization) -- **Run quality checks** → [USAGE.md](../USAGE.md) (Quality Command) -- **Export to Hugo** → [USAGE.md](../USAGE.md) (Export Command) -- **Fix errors** → [FAQ.md](../FAQ.md) (Troubleshooting) -- **Add a new feature** → [DEVELOPER_GUIDE.md](../DEVELOPER_GUIDE.md) (Adding New Features) -- **Run tests** → [TESTING.md](../TESTING.md) (Running Tests) -- **Contribute code** → [CONTRIBUTING.md](../CONTRIBUTING.md) (Contribution Workflow) - -### By Role - -**New User:** -1. [README.md](../README.md) → [INSTALL.md](../INSTALL.md) → [CONFIGURATION.md](../CONFIGURATION.md) → [USAGE.md](../USAGE.md) - -**Contributor:** -1. [README.md](../README.md) → [CONTRIBUTING.md](../CONTRIBUTING.md) → [TESTING.md](../TESTING.md) - -**Developer (adding features):** -1. [DEVELOPER_GUIDE.md](../DEVELOPER_GUIDE.md) → [CONTRIBUTING.md](../CONTRIBUTING.md) → [TESTING.md](../TESTING.md) - ---- - -## 📦 Project Documentation Files - -### Top-Level Documentation (8 files) - -Located in project root: - -``` -RM11/ -├── README.md # Project overview & quick start -├── INSTALL.md # Installation guide (515 lines) -├── USAGE.md # CLI command reference (800+ lines) -├── CONFIGURATION.md # Configuration guide (1,000+ lines) -├── FAQ.md # Common questions (550+ lines) -├── CONTRIBUTING.md # Contribution guidelines (450+ lines) -├── TESTING.md # Testing guide (600+ lines) -├── CHANGELOG.md # Version history (350+ lines) -├── DEVELOPER_GUIDE.md # Developer guide (1,000+ lines) -└── AGENTS.md # Agent design patterns -``` - -**Total:** 5,865+ lines of documentation - -### docs/ Directory - -Project-specific documentation: - -``` -docs/ -├── README.md # This file - documentation index -├── USER_GUIDE.md # Comprehensive user guide (1,424 lines) -├── MVP_CHECKPOINT.md # Milestone 2 verification -├── PHASE_5_COMPLETION.md # Testing & Quality report -├── PHASE_6_COMPLETION.md # Documentation & Polish report -├── AI_AGENT_TODO.md # Project roadmap -├── Test_Coverage_Analysis.md -├── INTEGRATION_TESTING_SUMMARY.md -├── REAL_API_VERIFICATION.md -└── OPTIMIZATION_SUMMARY.md -``` - -### data_reference/ Directory - -RootsMagic 11 schema documentation (18 files): - -``` -data_reference/ -├── RM11_Schema_Reference.md # Complete schema -├── RM11_Date_Format.md # 24-char date encoding -├── RM11_Place_Format.md # Place hierarchy -├── RM11_FactTypes.md # Event types -├── RM11_BLOB_*.md # XML BLOB specs (3 files) -├── RM11_Biography_Best_Practices.md # 9-section structure -├── RM11_Data_Quality_Rules.md # 24 validation rules -├── RM11_Query_Patterns.md # SQL patterns -├── RM11_Timeline_Construction.md # TimelineJS3 format -└── ... (10 more reference docs) -``` - ---- - -## 🔍 Documentation Statistics - -**Total Documentation:** -- **Lines:** 7,289+ lines -- **Files:** 36 files (10 top-level + 10 docs/ + 16 data_reference/) -- **Coverage:** Installation, usage, configuration, development, schema, API - -**By Category:** -- **User Documentation:** 3,500+ lines (INSTALL, USAGE, CONFIGURATION, FAQ, USER_GUIDE) -- **Developer Documentation:** 2,050+ lines (DEVELOPER_GUIDE, CONTRIBUTING, TESTING) -- **Project Documentation:** 1,200+ lines (CHANGELOG, completion reports, roadmap) -- **Schema Reference:** 18 files (data_reference/) - ---- - -## 📝 Documentation Updates - -### Recent Updates (v0.2.0 - October 12, 2025) - -**New Documentation:** -- ✅ **DEVELOPER_GUIDE.md** - Comprehensive developer guide with architecture, API reference, adding features -- ✅ **Prompt Customization** - Complete section in CONFIGURATION.md with YAML examples -- ✅ **USER_GUIDE.md** - Updated to v0.2.0 with new prompt system - -**Updated Documentation:** -- ✅ USER_GUIDE.md - "Customizing Prompts" section updated for YAML system -- ✅ CONFIGURATION.md - Added 350+ line "Prompt Customization" section -- ✅ CHANGELOG.md - Phase 5 and Phase 6 entries - -### Version History - -- **v0.2.0** (2025-10-12): Phase 5 & 6 complete - Testing, Quality, Documentation -- **v0.1.0** (2025-10-10): MVP complete - All 8 CLI commands working -- **v0.0.3** (2025-10-09): Milestone 1 - Working prototype - ---- - -## 🤝 Contributing to Documentation - -Documentation improvements are always welcome! - -**To contribute documentation:** - -1. **Small fixes:** Edit directly and submit PR -2. **New sections:** Open issue first to discuss -3. **Style guide:** Follow existing format -4. **Examples:** Include practical code examples -5. **Testing:** Verify all commands work - -See [CONTRIBUTING.md](../CONTRIBUTING.md) for details. - ---- - -## 📞 Getting Help - -**Questions about documentation:** -- Open an issue: https://github.com/miams/rmagent/issues -- Discussions: https://github.com/miams/rmagent/discussions - -**Found a bug in documentation:** -- Report it: https://github.com/miams/rmagent/issues -- Include: Which file, what's wrong, suggested fix - ---- - -**Last Updated:** October 12, 2025 diff --git a/docs/MVP_CHECKPOINT.md b/docs/archive/checkpoints/mvp-checkpoint.md similarity index 100% rename from docs/MVP_CHECKPOINT.md rename to docs/archive/checkpoints/mvp-checkpoint.md diff --git a/docs/PHASE_5_COMPLETION.md b/docs/archive/checkpoints/phase-5-completion.md similarity index 100% rename from docs/PHASE_5_COMPLETION.md rename to docs/archive/checkpoints/phase-5-completion.md diff --git a/docs/PHASE_6_COMPLETION.md b/docs/archive/checkpoints/phase-6-completion.md similarity index 100% rename from docs/PHASE_6_COMPLETION.md rename to docs/archive/checkpoints/phase-6-completion.md diff --git a/biography.md b/docs/archive/summaries/biography-notes.md similarity index 100% rename from biography.md rename to docs/archive/summaries/biography-notes.md diff --git a/docs/CLI_SETUP.md b/docs/archive/summaries/cli-setup.md similarity index 100% rename from docs/CLI_SETUP.md rename to docs/archive/summaries/cli-setup.md diff --git a/docs/INTEGRATION_TESTING_SUMMARY.md b/docs/archive/summaries/integration-testing-summary.md similarity index 100% rename from docs/INTEGRATION_TESTING_SUMMARY.md rename to docs/archive/summaries/integration-testing-summary.md diff --git a/data_reference/RM11_Documentation_Index.md b/docs/archive/summaries/old-documentation-index.md similarity index 100% rename from data_reference/RM11_Documentation_Index.md rename to docs/archive/summaries/old-documentation-index.md diff --git a/docs/USER_GUIDE.md b/docs/archive/summaries/old-user-guide.md similarity index 100% rename from docs/USER_GUIDE.md rename to docs/archive/summaries/old-user-guide.md diff --git a/docs/OPTIMIZATION_SUMMARY.md b/docs/archive/summaries/optimization-summary.md similarity index 100% rename from docs/OPTIMIZATION_SUMMARY.md rename to docs/archive/summaries/optimization-summary.md diff --git a/docs/REAL_API_VERIFICATION.md b/docs/archive/summaries/real-api-verification.md similarity index 100% rename from docs/REAL_API_VERIFICATION.md rename to docs/archive/summaries/real-api-verification.md diff --git a/docs/SEARCH_LOGIC_FIX.md b/docs/archive/summaries/search-logic-fix.md similarity index 100% rename from docs/SEARCH_LOGIC_FIX.md rename to docs/archive/summaries/search-logic-fix.md diff --git a/docs/SETUP_COMPLETE.md b/docs/archive/summaries/setup-complete.md similarity index 100% rename from docs/SETUP_COMPLETE.md rename to docs/archive/summaries/setup-complete.md diff --git a/docs/Test_Coverage_Analysis.md b/docs/archive/summaries/test-coverage-analysis.md similarity index 100% rename from docs/Test_Coverage_Analysis.md rename to docs/archive/summaries/test-coverage-analysis.md diff --git a/docs/VALIDATION_RESULTS.md b/docs/archive/summaries/validation-results.md similarity index 100% rename from docs/VALIDATION_RESULTS.md rename to docs/archive/summaries/validation-results.md diff --git a/FAQ.md b/docs/faq.md similarity index 100% rename from FAQ.md rename to docs/faq.md diff --git a/CONFIGURATION.md b/docs/getting-started/configuration.md similarity index 100% rename from CONFIGURATION.md rename to docs/getting-started/configuration.md diff --git a/INSTALL.md b/docs/getting-started/installation.md similarity index 100% rename from INSTALL.md rename to docs/getting-started/installation.md diff --git a/QUICKSTART.md b/docs/getting-started/quickstart.md similarity index 100% rename from QUICKSTART.md rename to docs/getting-started/quickstart.md diff --git a/docs/guides/claude-code-setup.md b/docs/guides/claude-code-setup.md new file mode 100644 index 0000000..48f5d6a --- /dev/null +++ b/docs/guides/claude-code-setup.md @@ -0,0 +1,438 @@ +# Claude Code Setup for RMAgent + +This guide documents the slash commands and hooks configured for the RMAgent project. + +## Slash Commands + +All slash commands are defined in `.claude/commands/`. RMAgent-specific commands use the `rm-` prefix. + +**Total Commands: 12** (6 RMAgent-specific, 3 development, 3 utility) + +### RMAgent Commands (AI & Data) + +Commands that interact with RootsMagic database and AI features: + +#### `/rm-bio [length]` +**Description:** Generate a biography for a person using AI + +**Options:** +- `show-prompt: true` - Shows the AI prompt +- `meta: true` - Shows token usage and timing + +**Usage:** +``` +/rm-bio 1 # Standard biography +/rm-bio 1 short # Short biography +/rm-bio 1 comprehensive # Comprehensive biography +``` + +**Output:** Markdown file in `reports/biographies/` + +--- + +#### `/rm-ask ""` +**Description:** Ask natural language questions about genealogy data using AI + +**Options:** +- `show-prompt: true` - Shows the AI prompt +- `meta: true` - Shows token usage and timing + +**Usage:** +``` +/rm-ask "Who are the ancestors of John Smith?" +/rm-ask "Find everyone born in Baltimore" +/rm-ask "Show family relationships for person 1" +``` + +**Output:** AI-generated answer with data from database + +--- + +#### `/rm-person [options]` +**Description:** Query person details from RootsMagic database + +**Usage:** +``` +/rm-person 1 # Basic info +/rm-person 1 --events # Include all events +/rm-person 1 --family # Include family relationships +/rm-person 1 --ancestors # Include ancestors +/rm-person 1 --descendants # Include descendants +``` + +**Output:** Formatted person information with requested details + +--- + +#### `/rm-search [--name "Name"] [--place "Place"] [--limit N]` +**Description:** Search database by name or place + +**Usage:** +``` +/rm-search --name "John Smith" +/rm-search --place "Baltimore" +/rm-search --name "Smith" --limit 10 +``` + +**Output:** List of matching persons with IDs + +--- + +#### `/rm-quality [category]` +**Description:** Run data quality validation + +**Categories:** +- `dates` - Date format and range issues +- `names` - Missing or invalid names +- `places` - Place format issues +- `relationships` - Relationship inconsistencies +- `sources` - Citation and source issues +- `events` - Event data issues + +**Usage:** +``` +/rm-quality # All checks +/rm-quality dates # Only date issues +/rm-quality names # Only name issues +``` + +**Output:** Formatted table of data quality issues + +--- + +#### `/rm-timeline [options]` +**Description:** Generate timeline visualization + +**Usage:** +``` +/rm-timeline 1 # JSON format +/rm-timeline 1 --format html # HTML format +/rm-timeline 1 --include-family # Include family events +/rm-timeline 1 --group-by-phase # Group by life phases +``` + +**Output:** TimelineJS3 JSON or HTML file + +--- + +### Development Commands + +Generic development and testing commands (no `rm-` prefix): + +#### `/test [filter]` +**Description:** Run pytest tests with optional filters + +**Usage:** +``` +/test # Run all tests +/test unit # Run unit tests only +/test integration # Run integration tests only +/test test_queries.py # Run specific test file +``` + +**Output:** Test results with pass/fail status + +--- + +#### `/coverage` +**Description:** Run tests with coverage analysis + +**Usage:** +``` +/coverage +``` + +**Output:** Coverage report with line-by-line analysis. HTML report at `htmlcov/index.html` + +--- + +#### `/lint [fix]` +**Description:** Run code quality checks (ruff + black) + +**Usage:** +``` +/lint # Check only +/lint fix # Auto-fix issues +``` + +**Output:** Linting errors and warnings + +--- + +### Utility Commands + +#### `/docs [topic]` +**Description:** Quick access to documentation + +**Usage:** +``` +/docs # Show INDEX.md +/docs schema # Schema reference +/docs data-formats # Data formats reference +/docs dev # Developer guide +``` + +**Output:** Documentation preview (first 50-80 lines) + +--- + +#### `/doc-review [mode]` +**Description:** Review documentation for accuracy and completeness using AI + +**Options:** +- `show-prompt: true` - Shows the AI prompt +- `meta: true` - Shows token usage and timing + +**Usage:** +``` +/doc-review # Brief mode: review root docs + INDEX.md +/doc-review brief # Same as above +/doc-review deep # Deep mode: review ALL documentation files +``` + +**Brief Mode Reviews:** +- CLAUDE.md, README.md, AGENTS.md, CONTRIBUTING.md, CHANGELOG.md +- docs/INDEX.md +- Verifies INDEX.md accurately references all docs + +**Deep Mode Reviews:** +- All files from brief mode +- All documentation in docs/ directory +- Cross-references and consistency checks + +**Output:** AI assessment with critical issues, recommendations, and specific fixes + +--- + +#### `/check-db` +**Description:** Verify database file exists and is accessible + +**Usage:** +``` +/check-db +``` + +**Output:** Database path, status, and basic statistics (person/event/source counts) + +--- + +## Claude Code Hooks + +Hooks are configured in `.claude/settings.local.json` and run automatically at specific events. + +### PostToolUse Hooks + +#### Coverage Reminder Hook +**Trigger:** After running pytest with coverage (`uv run pytest --cov`) + +**Action:** Extracts coverage percentage and reminds to update CLAUDE.md and README.md if significantly changed + +**Output:** +``` +📊 Test coverage: 88% +💡 Reminder: Update coverage stats in CLAUDE.md and README.md if significantly changed +``` + +--- + +### PreToolUse Hooks + +#### Git Push Confirmation Hook +**Trigger:** Before git push commands + +**Action:** Shows recent commits being pushed + +**Output:** +``` +⚠️ Pushing to remote. Recent commits: +66a29ca docs: reorganize documentation structure +4cdea76 fix: constrain caption width to match image margins +528f1ef fix: handle sqlite3.Row objects in biography rendering +``` + +--- + +## Git Pre-Commit Hook + +A standard git pre-commit hook is configured at `.git/hooks/pre-commit` to remind about documentation updates. + +**Triggers:** +- Documentation structure changes → Check CLAUDE.md +- Test modifications → Check README.md for coverage stats +- Core code changes → Check CLAUDE.md +- Dependency changes → Check README.md + +**Output:** +``` +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +⚠️ DOCUMENTATION REVIEW REMINDER +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +This commit may require updates to key documentation files: + + • Documentation structure changed + +Please review and update if necessary: + + ✓ CLAUDE.md - Already modified + 📄 README.md - User-facing docs, badges, quick start + 📄 AGENTS.md - LangChain patterns, agent architecture + +Continue with commit? (y/n) +``` + +--- + +## Configuration Files + +### `.claude/settings.local.json` +- **Permissions:** Pre-approved bash commands, web fetch domains +- **Hooks:** PostToolUse and PreToolUse hook configurations + +### `.claude/commands/` +- All custom slash command definitions +- Markdown files with frontmatter and bash scripts + +### `.git/hooks/pre-commit` +- Git pre-commit hook for documentation review +- Executable shell script + +--- + +## Best Practices + +### When to Use Each Command + +**For Genealogy Work:** +- Use `/rm-person` to quickly explore individuals +- Use `/rm-search` to find people by name/place +- Use `/rm-bio` when you need a formatted biography +- Use `/rm-ask` for complex queries that need AI interpretation +- Use `/rm-quality` regularly to maintain data integrity + +**For Development:** +- Use `/test` for quick test runs during development +- Use `/coverage` before committing to check test coverage +- Use `/lint` before committing to catch style issues +- Use `/check-db` when debugging database connection issues + +**For Documentation:** +- Use `/docs` to quickly reference schema or data formats +- Use `/doc-review` regularly to ensure docs stay accurate and current +- Use `/doc-review brief` before major commits or PRs +- Use `/doc-review deep` after significant feature additions +- Keep CLAUDE.md, README.md, and AGENTS.md updated (hooks will remind you) + +### show-prompt and meta Flags + +Only use `show-prompt` and `meta` in commands that interact with LLMs: +- ✅ `/rm-bio` - Generates biography with AI +- ✅ `/rm-ask` - Uses AI to answer questions +- ✅ `/doc-review` - Uses AI to review documentation +- ❌ `/rm-person` - Pure database query +- ❌ `/rm-search` - Pure database query +- ❌ `/rm-quality` - Rule-based validation +- ❌ `/rm-timeline` - Data extraction and formatting + +### Naming Conventions + +- **RMAgent-specific commands:** Use `rm-` prefix (`/rm-bio`, `/rm-person`) +- **Generic dev commands:** No prefix (`/test`, `/lint`, `/coverage`) +- **Utility commands:** No prefix (`/docs`, `/check-db`) + +--- + +## Testing Your Setup + +After configuring slash commands and hooks: + +1. **Test a slash command:** + ``` + /check-db + ``` + +2. **Test an AI command:** + ``` + /doc-review brief + ``` + +3. **Test a hook:** Run pytest with coverage to see PostToolUse hook: + ```bash + uv run pytest --cov=rmagent + ``` + +4. **Test git hook:** Make a change to tests and try to commit: + ```bash + # Modify a test file + git add tests/ + git commit -m "test: update test" + # You should see the documentation review reminder + ``` + +5. **View all commands:** + ``` + /help + ``` + +--- + +## Troubleshooting + +### Slash commands not appearing +- Check that files exist in `.claude/commands/` +- Ensure markdown files have proper frontmatter +- Restart Claude Code + +### Hooks not firing +- Verify JSON syntax in `.claude/settings.local.json` +- Check that regex matchers are properly escaped +- Use `claude --debug` for detailed hook execution info + +### Git hook not running +- Verify `.git/hooks/pre-commit` exists and is executable: + ```bash + chmod +x .git/hooks/pre-commit + ``` +- Check that it's not being bypassed with `--no-verify` + +--- + +## Extending This Setup + +### Adding New Slash Commands + +1. Create `.claude/commands/your-command.md`: + ```markdown + --- + description: Your command description + show-prompt: true # Only if uses AI + meta: true # Only if uses AI + --- + + Usage instructions here. + + ```bash + # Your bash script + echo "Command output" + ``` + ``` + +2. Use `rm-` prefix if RMAgent-specific +3. Document in this file + +### Adding New Hooks + +1. Edit `.claude/settings.local.json` +2. Add to appropriate hook type (PreToolUse, PostToolUse, etc.) +3. Use regex matcher to target specific tools +4. Test thoroughly before committing + +--- + +## References + +- **Claude Code Docs:** https://docs.claude.com/en/docs/claude-code/ +- **Slash Commands:** https://docs.claude.com/en/docs/claude-code/slash-commands +- **Hooks:** https://docs.claude.com/en/docs/claude-code/hooks +- **RMAgent User Guide:** [guides/user-guide.md](user-guide.md) +- **Developer Guide:** [guides/developer-guide.md](developer-guide.md) diff --git a/DEVELOPER_GUIDE.md b/docs/guides/developer-guide.md similarity index 100% rename from DEVELOPER_GUIDE.md rename to docs/guides/developer-guide.md diff --git a/docs/GIT_WORKFLOW_GUIDE.md b/docs/guides/git-workflow.md similarity index 65% rename from docs/GIT_WORKFLOW_GUIDE.md rename to docs/guides/git-workflow.md index 0d52c68..bd76ec8 100644 --- a/docs/GIT_WORKFLOW_GUIDE.md +++ b/docs/guides/git-workflow.md @@ -20,9 +20,22 @@ feature/* Individual features ### Branches Explained -- **main**: Production-ready code only. Protected - no direct commits allowed. -- **develop**: Default working branch. All features merge here first. +- **main**: Production-ready code only. **Protected** - no direct commits allowed. + - Requires PR with 1 approving review + - Requires all status checks to pass (linting + tests) + - Requires branch to be up-to-date before merge + - No force pushes or deletions allowed + - Enforced for administrators + +- **develop**: Default working branch. **Protected** - all features merge here first. + - Requires PR (self-merge allowed for solo dev) + - Requires all status checks to pass (linting + tests) + - No force pushes or deletions allowed + - Admins can bypass in emergencies + - **feature/**: Individual features. Branch from develop, PR back to develop. + - No protection - you can commit directly and force push if needed + - Still requires PR to merge into develop ## Daily Workflow @@ -75,6 +88,26 @@ git push - `refactor:` - Code refactoring - `chore:` - Maintenance tasks +**Pre-Commit Hook** (Automatic): + +When you commit, a pre-commit hook automatically checks if key documentation files need updating: + +``` +⚠️ DOCUMENTATION REVIEW REMINDER + +This commit may require updates to key documentation files: + + • Documentation structure changed + +Please review and update if necessary: + + 📄 CLAUDE.md - Project overview, structure, key patterns + 📄 README.md - User-facing docs, badges, quick start + ✓ AGENTS.md - Already modified +``` + +The hook will ask you to confirm before proceeding. This ensures CLAUDE.md, README.md, and AGENTS.md stay current. + ### 3. Creating a Pull Request When your feature is ready: @@ -91,10 +124,12 @@ gh pr create --base develop --title "feat: add census extraction" --body "Descri Or create the PR through GitHub web interface. **PR Checklist**: -- [ ] All tests pass locally (`uv run pytest`) -- [ ] Code is formatted (`uv run black .`) -- [ ] No linting errors (`uv run ruff check .`) +- [ ] All tests pass locally (`uv run pytest` or `/test`) +- [ ] Code is formatted (`uv run black .` or `/lint fix`) +- [ ] No linting errors (`uv run ruff check .` or `/lint`) - [ ] Changes are documented (if needed) +- [ ] Pre-commit hook reviewed (auto-runs on commit) +- [ ] CLAUDE.md, README.md, AGENTS.md updated if needed ### 4. Merging Your PR @@ -193,13 +228,71 @@ git commit --amend --no-edit git push --force ``` -## GitHub Actions CI/CD +## Automated Checks & Hooks + +RMAgent has three layers of automated checks: + +### 1. Git Pre-Commit Hook (Local) + +**Location:** `.git/hooks/pre-commit` + +**Triggers when:** +- Documentation structure changes → Reminds to check CLAUDE.md +- Test modifications → Reminds to check README.md for coverage stats +- Core code changes → Reminds to check CLAUDE.md +- Dependency updates → Reminds to check README.md + +**What it does:** +- Detects changes that might affect key documentation +- Shows which docs need review (with checkmarks for already-modified files) +- Prompts to confirm before allowing commit +- Prevents accidental outdated documentation + +**Example:** +``` +⚠️ DOCUMENTATION REVIEW REMINDER + + • Tests modified (check if coverage stats need updating) + +Please review and update if necessary: + + ✓ CLAUDE.md - Already modified + 📄 README.md - User-facing docs, badges, quick start + 📄 AGENTS.md - LangChain patterns, agent architecture + +Continue with commit? (y/n) +``` + +### 2. Claude Code Hooks (Development) + +**Location:** `.claude/settings.local.json` + +**PostToolUse Hook** - After running pytest with coverage: +``` +📊 Test coverage: 88% +💡 Reminder: Update coverage stats in CLAUDE.md and README.md if significantly changed +``` + +**PreToolUse Hook** - Before git push: +``` +⚠️ Pushing to remote. Recent commits: +66a29ca docs: reorganize documentation structure +e3937b5 feat: add Claude Code slash commands +``` + +**See [`docs/guides/claude-code-setup.md`](claude-code-setup.md) for full hook documentation.** + +### 3. GitHub Actions CI/CD (Remote) + +**Location:** `.github/workflows/pr-tests.yml` Every PR automatically runs: 1. **Linting** - `ruff check` and `black --check` 2. **Tests** - Full test suite with coverage 3. **Coverage Check** - Must maintain 80%+ coverage +**Required for merge** due to branch protection on `develop` and `main`. + **If tests fail**: 1. Check the Actions tab on GitHub for error details 2. Fix the issues locally @@ -307,6 +400,9 @@ uv run pytest -vv -s 5. **Test before PR** - Don't rely on CI to catch basic issues 6. **One feature per branch** - Don't mix unrelated changes 7. **Delete merged branches** - Keep your branch list clean +8. **Trust the hooks** - Pre-commit and Claude Code hooks help maintain quality +9. **Update docs proactively** - Don't wait for the hook to remind you +10. **Use slash commands** - `/test`, `/lint`, `/coverage` for quick checks ## Resources diff --git a/TESTING.md b/docs/guides/testing-guide.md similarity index 100% rename from TESTING.md rename to docs/guides/testing-guide.md diff --git a/USAGE.md b/docs/guides/user-guide.md similarity index 100% rename from USAGE.md rename to docs/guides/user-guide.md diff --git a/docs/DATA_PARSING_TODO.md b/docs/projects/ai-agent/data-parsing-todo.md similarity index 100% rename from docs/DATA_PARSING_TODO.md rename to docs/projects/ai-agent/data-parsing-todo.md diff --git a/docs/RM_Features_using_Langchain.md b/docs/projects/ai-agent/langchain-features.md similarity index 100% rename from docs/RM_Features_using_Langchain.md rename to docs/projects/ai-agent/langchain-features.md diff --git a/docs/RM11_LangChain_Upgrade.md b/docs/projects/ai-agent/langchain-upgrade.md similarity index 100% rename from docs/RM11_LangChain_Upgrade.md rename to docs/projects/ai-agent/langchain-upgrade.md diff --git a/docs/MULTI_AGENT_PLAN.md b/docs/projects/ai-agent/multi-agent-plan.md similarity index 100% rename from docs/MULTI_AGENT_PLAN.md rename to docs/projects/ai-agent/multi-agent-plan.md diff --git a/docs/AI_AGENT_TODO.md b/docs/projects/ai-agent/roadmap.md similarity index 100% rename from docs/AI_AGENT_TODO.md rename to docs/projects/ai-agent/roadmap.md diff --git a/docs/RM11_TimelineTODO.md b/docs/projects/ai-agent/timeline-todo.md similarity index 100% rename from docs/RM11_TimelineTODO.md rename to docs/projects/ai-agent/timeline-todo.md diff --git a/docs/Biography_Citation_Implementation_Plan.md b/docs/projects/biography-citations/citation-implementation.md similarity index 100% rename from docs/Biography_Citation_Implementation_Plan.md rename to docs/projects/biography-citations/citation-implementation.md diff --git a/docs/MARRIED_NAME_SEARCH_OPTIMIZATION.md b/docs/projects/biography-citations/married-name-search.md similarity index 100% rename from docs/MARRIED_NAME_SEARCH_OPTIMIZATION.md rename to docs/projects/biography-citations/married-name-search.md diff --git a/docs/RM11_CensusExtraction_Architecture.md b/docs/projects/census-extraction/architecture.md similarity index 100% rename from docs/RM11_CensusExtraction_Architecture.md rename to docs/projects/census-extraction/architecture.md diff --git a/docs/RM11_CensusExtraction_Plan.md b/docs/projects/census-extraction/implementation-plan.md similarity index 100% rename from docs/RM11_CensusExtraction_Plan.md rename to docs/projects/census-extraction/implementation-plan.md diff --git a/data_reference/RM11_Biography_Best_Practices.md b/docs/reference/biography/biography-best-practices.md similarity index 100% rename from data_reference/RM11_Biography_Best_Practices.md rename to docs/reference/biography/biography-best-practices.md diff --git a/data_reference/RM11_Timeline_Construction.md b/docs/reference/biography/timeline-construction.md similarity index 100% rename from data_reference/RM11_Timeline_Construction.md rename to docs/reference/biography/timeline-construction.md diff --git a/data_reference/RM11_BLOB_CitationFields.md b/docs/reference/data-formats/blob-citation-fields.md similarity index 100% rename from data_reference/RM11_BLOB_CitationFields.md rename to docs/reference/data-formats/blob-citation-fields.md diff --git a/data_reference/RM11_BLOB_SourceFields.md b/docs/reference/data-formats/blob-source-fields.md similarity index 100% rename from data_reference/RM11_BLOB_SourceFields.md rename to docs/reference/data-formats/blob-source-fields.md diff --git a/data_reference/RM11_BLOB_SourceTemplateFieldDefs.md b/docs/reference/data-formats/blob-template-field-defs.md similarity index 100% rename from data_reference/RM11_BLOB_SourceTemplateFieldDefs.md rename to docs/reference/data-formats/blob-template-field-defs.md diff --git a/data_reference/RM11_Date_Format.md b/docs/reference/data-formats/date-format.md similarity index 100% rename from data_reference/RM11_Date_Format.md rename to docs/reference/data-formats/date-format.md diff --git a/data_reference/RM11_Date_Format.yaml b/docs/reference/data-formats/date-format.yaml similarity index 100% rename from data_reference/RM11_Date_Format.yaml rename to docs/reference/data-formats/date-format.yaml diff --git a/data_reference/RM11_FactTypes.md b/docs/reference/data-formats/fact-types.md similarity index 100% rename from data_reference/RM11_FactTypes.md rename to docs/reference/data-formats/fact-types.md diff --git a/data_reference/RM11_Place_Format.md b/docs/reference/data-formats/place-format.md similarity index 100% rename from data_reference/RM11_Place_Format.md rename to docs/reference/data-formats/place-format.md diff --git a/data_reference/RM11_Sentence_Templates.md b/docs/reference/data-formats/sentence-templates.md similarity index 100% rename from data_reference/RM11_Sentence_Templates.md rename to docs/reference/data-formats/sentence-templates.md diff --git a/data_reference/RM11_Data_Quality_Rules.md b/docs/reference/query-patterns/data-quality-rules.md similarity index 100% rename from data_reference/RM11_Data_Quality_Rules.md rename to docs/reference/query-patterns/data-quality-rules.md diff --git a/data_reference/RM11_Query_Patterns.md b/docs/reference/query-patterns/query-patterns.md similarity index 100% rename from data_reference/RM11_Query_Patterns.md rename to docs/reference/query-patterns/query-patterns.md diff --git a/data_reference/RM11_schema_annotated.sql b/docs/reference/schema/annotated-schema.sql similarity index 100% rename from data_reference/RM11_schema_annotated.sql rename to docs/reference/schema/annotated-schema.sql diff --git a/data_reference/RM11_DataDef.yaml b/docs/reference/schema/data-definitions.yaml similarity index 100% rename from data_reference/RM11_DataDef.yaml rename to docs/reference/schema/data-definitions.yaml diff --git a/data_reference/RM11_EventTable_Details.md b/docs/reference/schema/event-table-details.md similarity index 100% rename from data_reference/RM11_EventTable_Details.md rename to docs/reference/schema/event-table-details.md diff --git a/data_reference/RM11_Name_Display_Logic.md b/docs/reference/schema/name-display-logic.md similarity index 100% rename from data_reference/RM11_Name_Display_Logic.md rename to docs/reference/schema/name-display-logic.md diff --git a/data_reference/RM11_Relationships.md b/docs/reference/schema/relationships.md similarity index 100% rename from data_reference/RM11_Relationships.md rename to docs/reference/schema/relationships.md diff --git a/data_reference/RM11_Schema_Reference.md b/docs/reference/schema/schema-reference.md similarity index 100% rename from data_reference/RM11_Schema_Reference.md rename to docs/reference/schema/schema-reference.md diff --git a/data_reference/RM11_schema.json b/docs/reference/schema/schema.json similarity index 100% rename from data_reference/RM11_schema.json rename to docs/reference/schema/schema.json