diff --git a/.gitignore b/.gitignore index 10741b8..cc5e3c4 100644 --- a/.gitignore +++ b/.gitignore @@ -37,3 +37,6 @@ report.[0-9]_.[0-9]_.[0-9]_.[0-9]_.json .dora/ stress-test/* + +.beans/* +.beans.yml diff --git a/.pi/skills/toon/SKILL.md b/.pi/skills/toon/SKILL.md index bb3deea..da0de0a 100644 --- a/.pi/skills/toon/SKILL.md +++ b/.pi/skills/toon/SKILL.md @@ -1,6 +1,6 @@ --- name: toon -description: **Token-Oriented Object Notation** is a compact, human-readable encoding of the JSON data model that minimizes tokens and makes structure easy for models to follow. It's intended for *LLM input* as a drop-in, lossless representation of your existing JSON. +description: Token-Oriented Object Notation is a compact, human-readable encoding of the JSON data model that minimizes tokens and makes structure easy for models to follow. It's intended for LLM input as a drop-in, lossless representation of your existing JSON. --- ![TOON logo with step‑by‑step guide](./.github/og.png) diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..2d65db9 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,80 @@ +# CLAUDE.md + +dora is a CLI that converts SCIP indexes into a queryable SQLite database. AI agents use it to answer questions about large codebases without reading files or tracing imports manually. + +## Stack + +- Runtime: Bun +- Database: SQLite via `bun:sqlite` +- Language: TypeScript +- Protobuf parsing: `@bufbuild/protobuf` +- AST parsing: `web-tree-sitter` (on-demand, per file) +- Output format: TOON by default (`dora status`), JSON via `--json` (`dora status --json`) + +## Source layout + +``` +src/ +├── commands/ # one file per CLI command +├── converter/ # SCIP protobuf parser + SQLite converter +├── db/ # schema and all SQL queries +├── mcp/ # MCP server, tool definitions, handlers +├── schemas/ # Zod schemas and inferred types +├── tree-sitter/ # grammar discovery, parser, language registry +└── utils/ # config, errors, output formatting +``` + +## Two indexing layers + +**SCIP** — runs the configured indexer (e.g. `scip-typescript`), produces a `.scip` protobuf, converts it to SQLite. Gives you symbols, references, and file-to-file dependencies derived from actual import resolution. + +**Tree-sitter** — parses source files on-demand using wasm grammars. Covers what SCIP doesn't: function signatures, cyclomatic complexity, class hierarchy, code smells. Grammar discovery checks local `node_modules`, then global bun packages, then explicit config paths. + +## Database design + +Denormalized counts (`symbol_count`, `dependency_count`, `dependent_count`, `reference_count`) are pre-computed at index time. Most queries are index lookups, not aggregations. + +Local symbols (function parameters, closure variables) are flagged `is_local = 1` and filtered out by default. Symbol kinds are extracted from SCIP documentation strings since `scip-typescript` doesn't populate the kind field. + +Schema: `src/converter/schema.sql`. All queries: `src/db/queries.ts`. + +## Config file: `.dora/config.json` + +```json +{ + "root": "/absolute/path/to/repo", + "scip": ".dora/index.scip", + "db": ".dora/dora.db", + "commands": { + "index": "scip-typescript index --output .dora/index.scip" + }, + "lastIndexed": "2025-01-15T10:30:00Z", + "ignore": ["test/**", "**/*.generated.ts"], + "treeSitter": { + "grammars": { + "typescript": "/explicit/path/to/tree-sitter-typescript.wasm" + } + } +} +``` + +## Code conventions + +- Single object parameter — never multiple positional params +- No inline comments, no section separators, no file headers +- No `any` — use `unknown` or proper types +- Boolean variables prefixed with `is` or `has` +- Use `type` not `interface` +- No emojis +- Output JSON to stdout, errors to stderr as `{"error": "message"}`, exit 1 on error + +## Adding a tree-sitter language + +1. Create `src/tree-sitter/languages/mylang.ts` — export `functionQueryString`, `classQueryString`, `parseFunctionCaptures`, `parseClassCaptures` +2. Register in `src/tree-sitter/languages/registry.ts` with grammar name and extensions +3. Add tests in `test/tree-sitter/` — see `function-captures.test.ts` as the reference. Tests mock `Parser.QueryCapture[]` objects directly; no wasm or disk I/O needed. + +## Hooks (`.claude/settings.json`) + +- **Stop**: runs `dora index` in the background after each turn +- **SessionStart**: checks index health, prompts to init if missing diff --git a/CHANGELOG.md b/CHANGELOG.md index 216db48..de26af3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,34 @@ # @butttons/dora +## 2.0.0 + +### New Commands + +- `dora fn ` — list all functions in a file with cyclomatic complexity, LOC, parameter types, return type, async/export flags, and SCIP reference count. Sort by complexity, loc, or name. Filter by minimum complexity. +- `dora class ` — list all classes with inheritance, implemented interfaces, decorators, abstract flag, method list, and property count. Sort by name, method count, or total complexity. +- `dora smells ` — detect code smells: high cyclomatic complexity, long functions, too many parameters, god classes (too many methods), large classes (too many properties), and TODO/FIXME/HACK comments. All thresholds configurable. + +### Enhancements + +- `dora file` now includes per-file metrics (LOC, SLOC, comment lines, blank lines, function count, avg/max complexity) and a full function list when a tree-sitter grammar is available. +- `dora symbol` now enriches function and method results with cyclomatic complexity, parameter types, and return type from tree-sitter. +- `dora status` now reports grammar availability for every registered language instead of only the project's primary language. + +### Tree-sitter Integration + +- Grammar discovery checks project-local `node_modules`, global bun packages, and explicit paths in `.dora/config.json` under `treeSitter.grammars`. +- TypeScript, TSX, JavaScript, and JSX supported out of the box. Additional languages can be added by implementing a query module and registering it. +- Parser WASM is initialized once per process. Language grammars are cached after first load. Global node_modules path is resolved once and cached. +- A tree-sitter cookbook recipe (`dora cookbook show tree-sitter`) covers workflows, complexity thresholds, and pre-refactor checklists. + +### Bug Fixes + +- Nullish coalescing (`??`) was not counted in cyclomatic complexity. The operator is a `binary_expression` node in tree-sitter, not a standalone node type. +- `bun.lock` (text format, Bun 1.1+) was not recognized by workspace detection. Only `bun.lockb` was checked, causing wrong `scip-typescript` flags for newer Bun projects. +- TODO comment scanning in `dora smells` missed markers inside multi-line block comments. +- `dora fn` and `dora class` silently fell back to the default sort on unrecognized `--sort` values instead of throwing an error. +- Absolute paths in tree-sitter commands were built with string interpolation instead of `path.resolve`, which is fragile on Windows and with trailing slashes. + ## 1.7.0 ### Minor Changes diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 4d77c2f..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,1447 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Project Overview - -# Build: dora - Code Context CLI for AI Agents - -## Overview - -Build a CLI tool called `dora` using Bun and SQLite. It helps AI agents understand large codebases by parsing SCIP protobuf indexes directly. - -## Tech Stack - -- Runtime: Bun -- Database: SQLite (via bun:sqlite) -- Language: TypeScript -- Protobuf: @bufbuild/protobuf -- Indexer: scip-typescript (external dependency) - -## Documentation Website - -The `docs/` directory contains the documentation website for dora, built with Astro and deployed to https://dora-cli.dev. - -- **Tech Stack:** Astro 5.x, Tailwind CSS 4.x, Cloudflare Workers -- **Pages:** Landing page (with platform/language detection), full documentation, command reference, architecture guide -- **Features:** Dynamic installation instructions for 9+ languages, AI agent integration examples, responsive design -- **Deployment:** Cloudflare Workers via wrangler (`bun run deploy` in docs directory) - -See `docs/CLAUDE.md` for detailed documentation-specific guidance on maintaining and updating the website. - -## MCP Server - -dora includes an MCP (Model Context Protocol) server that exposes all dora commands as tools for AI assistants like Claude Desktop. - -- **Command:** `dora mcp` -- **Implementation:** `src/mcp.ts` - Uses @modelcontextprotocol/sdk -- **Transport:** stdio (foreground process) -- **Tool handlers:** `src/mcp/handlers.ts` - Routes MCP tool calls to dora commands -- **Metadata:** `src/mcp/metadata.ts` - Tool definitions and schemas - -The MCP server runs in the foreground and communicates via stdin/stdout. It cannot be daemonized because MCP requires active stdio communication with the client. - -## Code Style Guidelines - -### Comments - -This codebase follows strict comment guidelines to maintain clean, self-documenting code: - -**Rules:** - -1. **NO inline comments** - Code should be self-explanatory through clear function and variable names -2. **NO section separator comments** - Like `// ===== Section Name =====` or `// Query commands` -3. **NO file header comments** - Like `// dora symbol command` or `// Type definitions for dora CLI` -4. **Use ONLY valid JSDoc** - And only when clearly warranted for complex public APIs -5. **NO emojis** - Keep code and comments professional and emoji-free - -**Valid JSDoc Format:** - -```typescript -/** - * Find shortest path between two files using BFS - */ -export function findPath(from: string, to: string): string[] { - // Implementation -} -``` - -**Examples of Comments to AVOID:** - -```typescript -// BAD - Obvious inline comment -const limit = options.limit || 20; // Default limit is 20 - -// BAD - Section separator -// ===== Status Queries ===== - -// BAD - File header -// dora symbol command - -// BAD - Explaining obvious code -// Get the symbol ID -const symbolId = getSymbolId(name); - -// BAD - Trivial explanation -// Loop through results -for (const result of results) { - // Process each result - processResult(result); -} -``` - -**Examples of Comments to KEEP:** - -```typescript -/** - * GOOD - Valid JSDoc for complex function - * Find tightly coupled file pairs (bidirectional dependencies) - */ -export function getCoupledFiles(threshold: number): CoupledFiles[] { - // Implementation -} - -/** - * GOOD - JSDoc with parameters and return type - * @param db - Database connection - * @param path - File path to analyze - * @returns Array of dependency nodes with depth information - */ -export function getDependencies(db: Database, path: string): DependencyNode[] { - // Implementation -} -``` - -**When Comments ARE Warranted:** - -- Complex algorithms that aren't immediately obvious -- Non-obvious edge cases or workarounds -- Security-related warnings or considerations -- Performance optimizations that aren't self-evident - -**Note:** Generated files (like `scip_pb.ts`) are exempt from these rules. - -### Function Parameters - -Functions with more than one parameter must use a single object parameter instead of multiple positional parameters. - -**Rules:** - -1. **Single parameter functions** - Can use a simple parameter type -2. **Multiple parameters** - Must use a single object parameter with named properties - -**Good:** - -```typescript -// Single parameter - OK -function getSymbol(id: number): Symbol { - // Implementation -} - -// Multiple parameters - use object -function createDocument(params: { - path: string; - type: string; - content: string; - mtime: number; -}): Document { - // Implementation -} - -// Better with type alias -type CreateDocumentParams = { - path: string; - type: string; - content: string; - mtime: number; -}; - -function createDocument(params: CreateDocumentParams): Document { - // Implementation -} -``` - -**Bad:** - -```typescript -// BAD - Multiple positional parameters -function createDocument( - path: string, - type: string, - content: string, - mtime: number, -): Document { - // Implementation -} - -// BAD - Two or more parameters -function batchInsert( - db: Database, - table: string, - columns: string[], - rows: Array>, -): void { - // Implementation -} -``` - -## Directory Structure - -Note: Example scip files in `example` folder -Required tools: - -``` -scip-typescript # For generating SCIP indexes -``` - -``` -dora/ -├── src/ -│ ├── index.ts # CLI entry point (Commander) -│ ├── commands/ -│ │ ├── init.ts # Initialize dora -│ │ ├── index.ts # Reindex command -│ │ ├── status.ts # Show index status -│ │ ├── overview.ts # High-level statistics -│ │ ├── file.ts # File analysis -│ │ ├── symbol.ts # Symbol search -│ │ ├── deps.ts # Dependencies -│ │ ├── rdeps.ts # Reverse dependencies -│ │ ├── path.ts # Find path between files -│ │ ├── changes.ts # Changed/impacted files -│ │ ├── exports.ts # Exported symbols -│ │ ├── imports.ts # File imports -│ │ ├── leaves.ts # Leaf nodes -│ │ ├── refs.ts # Symbol references -│ │ ├── unused.ts # Unused symbols -│ │ ├── hotspots.ts # Most referenced files -│ │ ├── graph.ts # Dependency graph -│ │ ├── ls.ts # List files in directory -│ │ └── shared.ts # Shared utilities -│ ├── converter/ -│ │ ├── convert.ts # SCIP → SQLite converter -│ │ ├── scip-parser.ts # SCIP protobuf parser -│ │ ├── scip_pb.ts # Generated protobuf types -│ │ ├── helpers.ts # Symbol kind mappings -│ │ └── schema.sql # Database schema -│ ├── proto/ -│ │ └── scip.proto # SCIP protobuf schema -│ ├── db/ -│ │ ├── queries.ts # All SQL queries -│ │ └── connection.ts # Database setup -│ └── utils/ -│ ├── config.ts # Config read/write -│ ├── output.ts # JSON formatting -│ ├── errors.ts # Error handling -│ └── changeDetection.ts # Incremental indexing -├── package.json -└── tsconfig.json -``` - -## Config File: .dora/config.json - -```json -{ - "root": "/absolute/path/to/repo", - "scip": ".dora/index.scip", - "db": ".dora/dora.db", - "commands": { - "index": "scip-typescript index --output .dora/index.scip" - }, - "lastIndexed": "2025-01-15T10:30:00Z", - "ignore": ["test/**", "**/*.generated.ts"] -} -``` - -**Optional fields:** - -- `ignore` - Array of glob patterns for files to exclude from indexing (e.g., `["test/**", "**/*.d.ts"]`) - -## Hooks - -This project uses two Claude Code hooks configured in `.claude/settings.json`: - -### Stop hook - -Automatically runs `dora index` in the background after each AI turn to keep the index up-to-date. Logs output to `/tmp/dora-index.log`. Non-blocking and never fails. - -Command: `(dora index > /tmp/dora-index.log 2>&1 &) || true` - -### SessionStart hook - -Checks if dora is initialized when a new session starts. If not initialized, displays a hint to run `dora init && dora index`. - -Command: `dora status 2>/dev/null || echo 'dora not initialized. Run: dora init && dora index'` - -## Database Schema - -The doraCLI uses an optimized SQLite database with denormalized fields for high performance: - -### files - -- `id` - Primary key -- `path` - Relative path from repo root (UNIQUE) -- `language` - Programming language -- `mtime` - File modification time -- `symbol_count` - Number of symbols in file (denormalized) -- `indexed_at` - When file was indexed -- `dependency_count` - Number of outgoing dependencies (denormalized) -- `dependent_count` - Number of incoming dependencies / fan-in (denormalized) - -### symbols - -- `id` - Primary key -- `file_id` - Foreign key to files table -- `name` - Symbol name (e.g., "Logger", "UserContext") -- `scip_symbol` - Full SCIP symbol identifier -- `kind` - Symbol kind extracted from documentation (class, function, interface, property, method, parameter, variable, type, enum, etc.) -- `start_line`, `end_line` - Line range -- `start_char`, `end_char` - Character range -- `documentation` - Symbol documentation/comments -- `package` - Package name if external symbol -- `is_local` - Boolean flag for local symbols (function parameters, closure variables) - filtered by default -- `reference_count` - Number of references to this symbol (denormalized) - -### dependencies - -- `from_file_id` - File that imports -- `to_file_id` - File being imported -- `symbol_count` - Number of symbols used -- `symbols` - JSON array of symbol names used - -### symbol_references - -- `id` - Primary key -- `symbol_id` - Foreign key to symbols table -- `file_id` - File where symbol is referenced -- `line` - Line number of reference - -### packages - -- `id` - Primary key -- `name` - Package name (e.g., "@org/package") -- `manager` - Package manager (npm, yarn, etc.) -- `version` - Package version -- `symbol_count` - Number of symbols from this package - -### metadata - -- `key` - Metadata key -- `value` - Metadata value - -## Commands - -### dora init - -- Detect repo root (find nearest package.json or tsconfig.json) -- Create .dora/ directory -- Add .dora to .gitignore if not present -- Write initial config.json -- Output: success message - -Note: Hooks are configured in .claude/settings.json (Stop and SessionStart) - -### dora index - -- Run: `scip-typescript index --output .dora/index.scip` (if configured) -- Parse SCIP protobuf file directly using @bufbuild/protobuf -- Convert to optimized custom SQLite database -- Support incremental builds (only reindex changed files) -- Update lastIndexed in config -- **Flags:** - - `--full` - Force full rebuild (ignore incremental detection) - - `--skip-scip` - Skip running SCIP indexer, use existing .scip file - - `--ignore ` - Ignore files matching glob pattern (can be repeated) -- Output: file count, symbol count, time taken, mode (full/incremental) - -### dora status - -- Check if .dora/dora.db exists -- Query file count, symbol count -- Show lastIndexed timestamp -- Output: JSON with health info - -### dora map - -Provides high-level statistics about the codebase: packages, file count, and symbol count. - -Queries: - -```sql --- Packages -SELECT name -FROM packages -ORDER BY name; - --- File count -SELECT COUNT(*) as count FROM files; - --- Symbol count -SELECT COUNT(*) as count FROM symbols; -``` - -Output: - -```json -{ - "packages": ["@zomunk/api-worker", ...], - "file_count": 412, - "symbol_count": 58917 -} -``` - -**Note:** `dora map` provides basic statistics only. For detailed code exploration: - -- Use `dora symbol ` to find specific symbols -- Use `dora file ` to explore specific files with dependencies -- Use `dora deps`/`dora rdeps` to understand relationships - -### dora ls [directory] - -List files in a directory from the index with metadata. - -**Arguments:** - -- `[directory]` - Optional directory path to list. Omit to list all files. Uses SQL LIKE pattern matching (`directory/%`). - -**Flags:** - -- `--limit ` - Maximum number of results (default: 100) -- `--sort ` - Sort by: `path`, `symbols`, `deps`, or `rdeps` (default: `path`) - -**Query:** - -```sql -SELECT - f.path, - f.symbol_count as symbols, - f.dependency_count as dependencies, - f.dependent_count as dependents -FROM files f -WHERE f.path LIKE ? -ORDER BY [selected_field] -LIMIT ? -``` - -**Output:** - -```json -{ - "directory": "src/commands", - "files": [ - { - "path": "src/commands/shared.ts", - "symbols": 35, - "dependencies": 7, - "dependents": 18 - } - ], - "total": 27 -} -``` - -**Use Cases:** - -- Browse files in a specific directory: `dora ls src/components` -- Find files with most symbols: `dora ls --sort symbols --limit 10` -- Find files with most dependencies: `dora ls --sort deps --limit 20` -- Find hub files (most dependents): `dora ls --sort rdeps --limit 10` - -### dora file - -Queries: - -```sql --- Symbols in file -SELECT - s.name, - s.kind, - s.start_line, - s.end_line -FROM symbols s -JOIN files f ON f.id = s.file_id -WHERE f.path = ? -ORDER BY s.start_line; - --- Dependencies (files this file imports) -SELECT - f.path as depends_on, - d.symbols as symbols_used -FROM dependencies d -JOIN files f ON f.id = d.to_file_id -WHERE d.from_file_id = (SELECT id FROM files WHERE path = ?) -ORDER BY f.path; - --- Dependents (files that import this file) -SELECT - f.path as dependent, - d.symbol_count as ref_count -FROM dependencies d -JOIN files f ON f.id = d.from_file_id -WHERE d.to_file_id = (SELECT id FROM files WHERE path = ?) -ORDER BY d.symbol_count DESC; -``` - -Output: - -```json -{ - "path": "apps/api-worker/src/context.ts", - "symbols": [ - { - "name": "PlatformContext", - "kind": "type", - "lines": [6, 21] - } - ], - "depends_on": [ - { "path": "packages/app-auth/src/index.ts", "symbols": ["AuthSession"] } - ], - "depended_by": [ - { "path": "apps/api-worker/src/router/billing/router.ts", "refs": 81 } - ] -} -``` - -**Note:** Does NOT include source code. Use Read tool to get file contents. - -### dora symbol - -Search for symbols by name with automatic filtering of local symbols. - -Query: - -```sql -SELECT - s.name, - s.kind, - f.path, - s.start_line, - s.end_line -FROM symbols s -JOIN files f ON f.id = s.file_id -WHERE s.name LIKE '%' || ? || '%' - AND s.is_local = 0 -- Filter out local symbols (parameters, closure vars) -LIMIT ?; -``` - -Flags: --kind, --limit (default: 20) - -Output: - -```json -{ - "query": "Logger", - "results": [ - { - "name": "Logger", - "kind": "interface", - "path": "packages/app-utils/src/logger.ts", - "lines": [7, 11] - } - ] -} -``` - -**Note:** Symbol kinds are automatically extracted from SCIP documentation strings since scip-typescript doesn't populate the kind field. - -### dora deps [--depth N] - -Query: - -```sql -WITH RECURSIVE dep_tree AS ( - -- Base case: start with the target file - SELECT id, path, 0 as depth - FROM files - WHERE path = ? - - UNION - - -- Recursive case: find files that this file depends on - SELECT DISTINCT f.id, f.path, dt.depth + 1 - FROM dep_tree dt - JOIN dependencies d ON d.from_file_id = dt.id - JOIN files f ON f.id = d.to_file_id - WHERE dt.depth < ? -) -SELECT path, MIN(depth) as depth -FROM dep_tree -WHERE depth > 0 -GROUP BY path -ORDER BY depth, path; -``` - -Default depth: 1 - -Output: - -```json -{ - "path": "apps/api-worker/src/context.ts", - "depth": 2, - "dependencies": [ - { "path": "packages/app-auth/src/index.ts", "depth": 1 }, - { "path": "packages/app-db/src/data/index.ts", "depth": 1 }, - { "path": "packages/app-db/src/billing/client.ts", "depth": 2 } - ] -} -``` - -### dora rdeps [--depth N] - -Query: - -```sql -WITH RECURSIVE rdep_tree AS ( - -- Base case: start with the target file - SELECT id, path, 0 as depth - FROM files - WHERE path = ? - - UNION - - -- Recursive case: find files that depend on this file - SELECT DISTINCT f.id, f.path, rt.depth + 1 - FROM rdep_tree rt - JOIN dependencies d ON d.to_file_id = rt.id - JOIN files f ON f.id = d.from_file_id - WHERE rt.depth < ? -) -SELECT path, MIN(depth) as depth -FROM rdep_tree -WHERE depth > 0 -GROUP BY path -ORDER BY depth, path; -``` - -Default depth: 1 - -Output: - -```json -{ - "path": "apps/api-worker/src/context.ts", - "depth": 2, - "dependents": [ - { "path": "apps/api-worker/src/router/billing/router.ts", "depth": 1 }, - { "path": "apps/api-worker/src/index.ts", "depth": 2 } - ] -} -``` - -### dora adventure - -Find shortest path between two files using BFS on the dependency graph. - -Query both deps and rdeps, find intersection. - -Output: - -```json -{ - "from": "apps/api-worker/src/router/billing/router.ts", - "to": "packages/app-utils/src/logger.ts", - "path": [ - "apps/api-worker/src/router/billing/router.ts", - "apps/api-worker/src/context.ts", - "packages/app-utils/src/logger.ts" - ], - "distance": 2 -} -``` - -## Documentation Commands - -### dora docs [--type TYPE] - -List all indexed documentation files. - -**Purpose:** Discover what documentation exists in the project. Useful for AI agents to understand what documentation is available before searching or exploring. - -**Flags:** - -- `--type ` - Filter by document type (md, txt) - -**Query:** - -```sql -SELECT path, type, symbol_count, file_count, document_count -FROM documents -ORDER BY path; -``` - -**Output:** - -```json -{ - "documents": [ - { - "path": "README.md", - "type": "markdown", - "symbol_refs": 12, - "file_refs": 4, - "document_refs": 2 - }, - { - "path": "docs/api.md", - "type": "markdown", - "symbol_refs": 8, - "file_refs": 3, - "document_refs": 0 - } - ], - "total": 2 -} -``` - -**Use Cases:** - -- Discover what documentation files exist in the project -- Filter documentation by type (markdown vs plain text) -- Quick overview of documentation coverage - -**Note:** To find documentation about specific code, use `dora symbol` or `dora file` which include a `documented_in` field showing which docs reference that code. - ---- - -### dora docs search - -Search through all indexed documentation files for specific text content. - -**Purpose:** Full-text search across all documentation. Useful for finding mentions of concepts, keywords, or specific phrases in your project's documentation. - -**Flags:** - -- `--limit ` - Maximum number of results to return (default: 20) - -**Query:** - -```sql -SELECT - d.path, - d.type, - d.symbol_count, - d.file_count -FROM documents d -WHERE d.content LIKE '%' || ? || '%' -ORDER BY d.path -LIMIT ?; -``` - -**Output:** - -```json -{ - "query": "authentication", - "limit": 20, - "results": [ - { - "path": "docs/api.md", - "type": "markdown", - "symbol_refs": 8, - "file_refs": 3 - }, - { - "path": "docs/setup.md", - "type": "markdown", - "symbol_refs": 2, - "file_refs": 1 - } - ], - "total": 2 -} -``` - -**Use Cases:** - -- Finding documentation about a specific topic -- Searching for configuration examples -- Locating documentation that needs updating -- Discovering related documentation across the project - ---- - -### dora docs show - -Display metadata and references for a specific documentation file. - -**Purpose:** Understand what a documentation file covers by showing which symbols and files it references, along with line numbers where references occur. - -**Flags:** - -- `--content` - Include the full document content in the output - -**Queries:** - -```sql --- Get document metadata -SELECT path, type, content -FROM documents -WHERE path = ?; - --- Get symbol references with line numbers -SELECT - s.name, - s.kind, - f.path, - s.start_line, - s.end_line, - dsr.line as ref_line -FROM document_symbol_refs dsr -JOIN symbols s ON s.id = dsr.symbol_id -JOIN files f ON f.id = s.file_id -WHERE dsr.document_id = ? AND s.name != '' -ORDER BY dsr.line; - --- Get file references with line numbers -SELECT - f.path, - dfr.line as ref_line -FROM document_file_refs dfr -JOIN files f ON f.id = dfr.file_id -WHERE dfr.document_id = ? -ORDER BY dfr.line; -``` - -**Output (without --content):** - -```json -{ - "path": "docs/authentication.md", - "type": "markdown", - "symbol_refs": [ - { - "name": "AuthService", - "kind": "class", - "path": "src/auth/service.ts", - "lines": [15, 42], - "ref_line": 23 - }, - { - "name": "validateToken", - "kind": "function", - "path": "src/auth/token.ts", - "lines": [8, 12], - "ref_line": 45 - } - ], - "file_refs": [ - { - "path": "src/auth/config.ts", - "ref_line": 12 - } - ] -} -``` - -**Output (with --content):** - -```json -{ - "path": "docs/authentication.md", - "type": "markdown", - "symbol_refs": [...], - "file_refs": [...], - "content": "# Authentication\n\nThis document describes..." -} -``` - -**Use Cases:** - -- Understanding what code a documentation file covers -- Finding exact line numbers where symbols/files are mentioned -- Verifying documentation accuracy against code -- Reviewing documentation coverage for specific features - ---- - -**What Files Are Indexed:** - -The documentation indexer automatically processes these file types: - -- `.md` - Markdown files -- `.txt` - Plain text documentation - -**Exclusions:** - -- Respects `.gitignore` patterns -- Auto-ignores: `node_modules/`, `.git/`, `.dora/`, `dist/`, `build/`, `coverage/`, `.next/`, `.nuxt/`, `out/`, `*.log` - -**Integration with Other Commands:** - -Documentation references are automatically included in: - -- `dora status` - Shows document count and breakdown by type -- `dora symbol ` - Shows which docs mention the symbol (via `documented_in` field) -- `dora file ` - Shows which docs reference the file (via `documented_in` field) - -## Architecture Analysis Commands - -### dora cycles [--limit N] - -Find bidirectional dependencies (files that import each other). - -**Purpose:** Identify 2-node circular dependencies where A imports B and B imports A. These are the most common and impactful architectural code smells. - -**Note:** This command only detects 2-node cycles. For longer cycles (A → B → C → A), use the `dora query` command with custom SQL. - -**Default:** `--limit 50` - -Query: - -```sql -SELECT - f1.path as path1, - f2.path as path2 -FROM dependencies d1 -JOIN dependencies d2 ON d1.from_file_id = d2.to_file_id - AND d1.to_file_id = d2.from_file_id -JOIN files f1 ON f1.id = d1.from_file_id -JOIN files f2 ON f2.id = d1.to_file_id -WHERE f1.path < f2.path -- avoid duplicates -ORDER BY f1.path, f2.path -LIMIT ?; -``` - -Output: - -```json -{ - "cycles": [ - { - "files": [ - "src/billing.ts", - "src/billing-subscription.ts", - "src/billing.ts" - ], - "length": 2 - } - ] -} -``` - -**Interpretation:** - -- Empty result = No bidirectional dependencies -- Cycles found = Refactor to break the cycle (extract shared types, merge files, or make dependency one-way) - -**Related:** Use `dora coupling` to see how many symbols are shared between bidirectional dependencies. - ---- - -### dora coupling [--threshold N] - -Find tightly coupled file pairs (bidirectional dependencies). - -**Purpose:** Identify files that import symbols from each other, indicating potential for refactoring. - -Query: - -```sql -SELECT - f1.path as file1, - f2.path as file2, - d1.symbol_count as symbols_1_to_2, - d2.symbol_count as symbols_2_to_1, - (d1.symbol_count + d2.symbol_count) as total_coupling -FROM dependencies d1 -JOIN dependencies d2 ON d1.from_file_id = d2.to_file_id - AND d1.to_file_id = d2.from_file_id -JOIN files f1 ON f1.id = d1.from_file_id -JOIN files f2 ON f2.id = d1.to_file_id -WHERE f1.path < f2.path - AND (d1.symbol_count + d2.symbol_count) >= ? -ORDER BY total_coupling DESC; -``` - -Default threshold: 5 - -Output: - -```json -{ - "threshold": 5, - "coupled_files": [ - { - "file1": "src/billing.ts", - "file2": "src/billing-subscription.ts", - "symbols_1_to_2": 2, - "symbols_2_to_1": 1, - "total_coupling": 3 - } - ] -} -``` - -**Interpretation:** - -- Low coupling (< 5) = Files share a few types, normal -- High coupling (> 20) = Consider merging or extracting shared module - ---- - -### dora complexity [--sort metric] - -Show file complexity metrics for refactoring prioritization. - -**Purpose:** Identify files that are risky to change based on size, dependencies, and impact. - -Query: - -```sql -SELECT - f.path, - f.symbol_count, - f.dependency_count as outgoing_deps, - f.dependent_count as incoming_deps, - CAST(f.dependent_count AS FLOAT) / NULLIF(f.dependency_count, 1) as stability_ratio, - (f.symbol_count * f.dependent_count) as complexity_score -FROM files f -ORDER BY [selected_metric] DESC -LIMIT 20; -``` - -Flags: --sort (complexity | symbols | stability) - -**Metrics:** - -- `symbol_count` - Proxy for lines of code -- `outgoing_deps` - Files this file imports from -- `incoming_deps` - Files that import from this file (fan-in) -- `stability_ratio` - incoming / outgoing (high = stable, hard to change) -- `complexity_score` - symbols × incoming deps (high = risky to change) - -Output: - -```json -{ - "sort_by": "complexity", - "files": [ - { - "path": "src/types.ts", - "symbol_count": 180, - "outgoing_deps": 0, - "incoming_deps": 52, - "stability_ratio": null, - "complexity_score": 9360 - } - ] -} -``` - -**Interpretation:** - -- High complexity score (> 5000) = High-impact file, changes affect many files -- High stability ratio (> 5) = Stable interface, expensive to change -- Low incoming deps (< 3) = Good refactoring candidate - ---- - -## Additional Commands - -### dora refs - -Find all references to a symbol across the codebase. - -**Performance:** Uses single optimized query with GROUP_CONCAT (no N+1 queries). - -Output includes definition location and all files that reference the symbol. - ---- - -### dora lost [--limit N] - -Find potentially unused symbols (zero references). - -**Performance:** Uses denormalized `reference_count` field for instant results. - -Query: - -```sql -SELECT s.name, s.kind, f.path, s.start_line, s.end_line -FROM symbols s -JOIN files f ON f.id = s.file_id -WHERE s.is_local = 0 - AND s.reference_count = 0 - AND s.kind NOT IN ('module', 'parameter') -ORDER BY f.path, s.start_line -LIMIT ?; -``` - ---- - -### dora treasure [--limit N] - -Show most referenced files and files with most dependencies. - -**Performance:** Uses denormalized `dependent_count` and `dependency_count` fields. - -Output: - -```json -{ - "most_referenced": [{ "file": "src/types.ts", "count": 52 }], - "most_dependencies": [{ "file": "src/app.tsx", "count": 27 }] -} -``` - ---- - -### dora changes [ref] - -Show files changed since git ref and their impact. - ---- - -### dora exports - -List exported symbols from a file or package. - ---- - -### dora imports - -Show all imports for a file. - ---- - -### dora leaves [--max-dependents N] - -Find leaf nodes (files with few dependents but have dependencies). - -**Default:** `--max-dependents 0` (files with zero dependents) - ---- - -### dora graph [--direction] [--depth] - -Generate dependency graph data. - ---- - -### dora schema - -Show the complete database schema including tables, columns, types, and indexes. - -**Purpose:** Provides the schema needed for AI agents to write custom SQL queries using `dora query`. - -Output: - -```json -{ - "tables": [ - { - "name": "files", - "columns": [ - { - "name": "id", - "type": "INTEGER", - "nullable": true, - "primary_key": true - }, - { - "name": "path", - "type": "TEXT", - "nullable": false, - "primary_key": false - } - ], - "indexes": ["CREATE INDEX idx_files_path ON files(path)"] - } - ] -} -``` - -**Key Tables:** - -- `files` - File metadata (path, language, mtime, symbol_count, dependency_count, dependent_count) -- `symbols` - Symbol definitions (name, kind, file_id, location, is_local, reference_count) -- `dependencies` - File-to-file dependencies (from_file_id, to_file_id, symbol_count, symbols) -- `symbol_references` - Symbol usage tracking (symbol_id, file_id, line) -- `packages` - External packages (name, manager, version, symbol_count) -- `metadata` - System metadata (key-value pairs) - ---- - -### dora cookbook show [recipe] - -Show query pattern cookbook with examples and tips for common SQL patterns. - -**Purpose:** Provides ready-to-use SQL query patterns for AI agents and users who want to explore the database without needing to learn the schema first. All recipes include real examples tested on actual codebases. - -**Flags:** - -- `[recipe]` - Optional recipe name. Omit to see all available recipes. - -**Available Recipes:** - -- `quickstart` - Complete walkthrough exploring a codebase from scratch with real-world workflows -- `methods` - Finding class methods by name, finding all methods in a class, counting method usages -- `references` - Tracking symbol usage, finding most referenced symbols, identifying dead code -- `exports` - Distinguishing exported symbols from internal ones, finding public API functions/types -- `agent-setup` - Setting up dora hooks, extensions, and skills for AI agents (Claude Code, pi, OpenCode, Cursor, Windsurf) - -**Output:** - -```json -{ - "recipe": "quickstart", - "content": "# Dora Quickstart: Exploring a Codebase\n\nA practical walkthrough..." -} -``` - -**Usage:** - -```bash -# Show all available recipes -dora cookbook list - -# Show quickstart guide -dora cookbook show quickstart - -# Show methods recipe -dora cookbook show methods - -# Show references recipe -dora cookbook show references - -# Show exports recipe -dora cookbook show exports -``` - -**Use Cases:** - -- **New to dora?** Start with `dora cookbook quickstart` for a complete walkthrough -- Discovering query patterns for common tasks -- Learning how to use `dora query` effectively -- Finding SQL examples for specific use cases -- Understanding how to query methods, references, and exported symbols - -**Integration with Other Commands:** - -- Use `dora schema` to understand the database structure -- Use `dora query` to execute the SQL patterns shown in recipes -- Copy-paste SQL examples directly from cookbook output into `dora query` - -**Custom Cookbooks:** -You can add your own cookbook recipes by creating markdown files in `.dora/cookbooks/`. Each file becomes a recipe that can be accessed with `dora cookbook show ` (without the .md extension). - -Example: Create `.dora/cookbooks/my-patterns.md` with your custom SQL patterns, then access it with `dora cookbook show my-patterns`. - ---- - -### dora query "" - -Execute arbitrary SQL queries against the database (read-only). - -**Purpose:** Enables ad-hoc analysis and custom queries not covered by built-in commands. AI agents can use this to explore the database and answer complex questions about the codebase. - -**Safety:** Only SELECT queries are allowed. INSERT, UPDATE, DELETE, DROP, CREATE, ALTER, TRUNCATE, and REPLACE operations are blocked. - -**Usage:** - -```bash -# Find files with most symbols -dora query "SELECT path, symbol_count FROM files ORDER BY symbol_count DESC LIMIT 10" - -# Count symbols by kind -dora query "SELECT kind, COUNT(*) as count FROM symbols WHERE is_local = 0 GROUP BY kind ORDER BY count DESC" - -# Find files with bidirectional dependencies -dora query "SELECT f1.path as file1, f2.path as file2 FROM dependencies d1 JOIN dependencies d2 ON d1.from_file_id = d2.to_file_id AND d1.to_file_id = d2.from_file_id JOIN files f1 ON f1.id = d1.from_file_id JOIN files f2 ON f2.id = d1.to_file_id WHERE f1.path < f2.path" - -# Analyze symbol distribution per file -dora query "SELECT f.path, COUNT(s.id) as symbols, AVG(s.reference_count) as avg_refs FROM files f JOIN symbols s ON s.file_id = f.id WHERE s.is_local = 0 GROUP BY f.path ORDER BY symbols DESC LIMIT 20" -``` - -Output: - -```json -{ - "query": "SELECT path, symbol_count FROM files ORDER BY symbol_count DESC LIMIT 5", - "rows": [ - { "path": "src/converter/scip_pb.ts", "symbol_count": 1640 }, - { "path": "src/proto/scip.proto", "symbol_count": 86 } - ], - "row_count": 2, - "columns": ["path", "symbol_count"] -} -``` - -**Tips for AI Agents:** - -- Use `dora schema` first to understand the database structure -- Filter local symbols with `WHERE is_local = 0` for cleaner results -- Use denormalized fields (`reference_count`, `dependent_count`, `dependency_count`) for fast queries -- JOIN tables to correlate symbols, files, and dependencies -- Use GROUP BY and aggregates (COUNT, SUM, AVG) for statistical analysis - -## CLI Entry Point - -The CLI uses the Commander library for argument parsing: - -```typescript -// src/index.ts (simplified) -import { Command } from "commander"; - -const program = new Command(); - -program - .name("dora") - .description("Code Context CLI for AI Agents") - .version("1.0.0"); - -// Setup commands -// init, index, status, overview - -// Query commands -// file, symbol, refs, deps, rdeps, path - -// Analysis commands -// cycles, coupling, complexity, hotspots, unused, leaves - -// Additional commands -// changes, exports, imports, graph - -program.parse(); -``` - -## Output Rules - -1. Always output valid JSON to stdout -2. Errors go to stderr as JSON: `{"error": "message"}` -3. No extra logging or formatting -4. Exit code 0 on success, 1 on error - -## Dependencies - -```json -{ - "name": "dora", - "type": "module", - "bin": { - "dora": "./src/index.ts" - }, - "dependencies": {}, - "devDependencies": { - "@types/bun": "latest", - "typescript": "^5.0.0" - } -} -``` - -## Debug Logging - -The CLI uses the [`debug`](https://www.npmjs.com/package/debug) library for logging. Control logging via the `DEBUG` environment variable: - -```bash -# Show all dora debug output -DEBUG=dora:* dora index - -# Show only converter logs -DEBUG=dora:converter dora index - -# Show multiple namespaces -DEBUG=dora:index,dora:converter dora index - -# Show all debug logs from all libraries -DEBUG=* dora index -``` - -Available namespaces: - -- `dora:index` - Index command logs -- `dora:converter` - SCIP to DB conversion logs -- `dora:db` - Database operation logs -- `dora:config` - Configuration logs - -## Performance Optimizations - -The doraCLI uses several optimization strategies for fast queries on large codebases: - -### 1. Denormalized Fields - -Pre-computed aggregates stored in the database for instant lookups: - -- `files.symbol_count` - Number of symbols per file -- `files.dependency_count` - Outgoing dependencies -- `files.dependent_count` - Incoming dependencies (fan-in) -- `symbols.reference_count` - Number of references to each symbol - -**Impact:** 10-50x faster queries. No expensive COUNT() aggregations at query time. - -### 2. Symbol Kind Extraction - -Since scip-typescript doesn't populate the `kind` field, we extract symbol kinds from documentation strings: - -- Pattern matching on documentation like `"interface Logger"`, `"(property) name: string"` -- Supports: class, interface, type, function, method, property, parameter, variable, enum, etc. -- Stored in indexed `symbols.kind` column for fast filtering - -### 3. Local Symbol Filtering - -Local symbols (function parameters, closure variables) are flagged with `is_local = 1`: - -- Reduces noise in symbol searches -- Indexed boolean column for fast filtering -- ~15-20% of symbols are local and filtered by default - -### 4. Optimized Queries - -- Symbol references: Single JOIN query with GROUP_CONCAT (no N+1 queries) -- Unused symbols: Index lookup on `reference_count = 0` -- Hotspots: Direct lookup on denormalized counts -- Cycles: Recursive CTE with visit tracking - -### 5. Incremental Indexing - -- Only reindex files that changed (based on mtime) -- Full reindex only when forced or on first run -- Denormalized fields updated after each indexing operation - -## Notes - -- The database schema is defined in `src/converter/schema.sql` and created in `src/converter/convert.ts` -- SCIP protobuf files are parsed directly using `@bufbuild/protobuf` -- Source text is NOT in the database - use the Read tool to get file contents -- All paths in the database are relative to repo root -- Symbol strings (SCIP identifiers) look like: `scip-typescript npm @package/name 1.0.0 src/\`file.ts\`/SymbolName#` -- The converter supports both full and incremental builds based on git state and file mtimes -- Database uses optimized schema with denormalized data for fast queries -- Pre-computed dependencies and symbol references for O(1) lookups - -## Typical Workflow for AI Agents - -```bash -# 1. Initialize in a TypeScript project -dora init - -# 2. Index the codebase -dora index - -# 3. Understand the codebase structure -dora map -dora treasure --limit 20 - -# 4. Find specific code -dora symbol AuthService -dora file src/auth/service.ts - -# 5. Analyze architecture -dora cycles # Check for circular dependencies -dora coupling --threshold 5 # Find tightly coupled files -dora complexity --sort complexity # Find high-impact files - -# 6. Impact analysis before changes -dora rdeps src/types.ts --depth 2 # What depends on this? -dora lost --limit 50 # Find dead code - -# 7. Navigate dependencies -dora deps src/app.ts --depth 2 -dora adventure src/component.tsx src/utils.ts - -# 8. Advanced custom queries -dora cookbook show methods # Learn how to query methods -dora query "" # Execute custom SQL queries -``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d5e4775..9f18431 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,120 +1,58 @@ # Contributing to dora -Thank you for your interest in contributing to dora! This document provides guidelines and information for developers. +## Development setup -## Getting Started - -### Prerequisites - -- [Bun](https://bun.sh) 1.0+ (for development) -- A SCIP-compatible indexer for your language (e.g., scip-typescript for TypeScript/JavaScript) - -### Development Setup +Requires [Bun](https://bun.sh) 1.0+. ```bash -# Clone repository git clone https://github.com/butttons/dora.git cd dora - -# Install dependencies bun install - -# Run CLI directly with Bun -bun src/index.ts - -# Run tests -bun test - -# Build and test the binary -bun run build -./dist/dora --help - -# Link for local development -bun link +bun link # makes `dora` point to src/index.ts via bun ``` -For detailed architecture and development guidelines, see [CLAUDE.md](./CLAUDE.md). - -## Building - -Build standalone binaries for distribution: +Run any command directly: ```bash -# Build for your current platform -bun run build - -# Build for specific platforms -bun run build:linux # Linux x64 -bun run build:macos # macOS Intel -bun run build:macos-arm # macOS ARM (M1/M2/M3) -bun run build:windows # Windows x64 - -# Build for all platforms -bun run build:all - -# Binaries will be in the dist/ directory +bun src/index.ts status +bun src/index.ts symbol AuthService ``` -Binary sizes: - -- **macOS/Linux**: ~57MB (includes Bun runtime) -- **Windows**: ~58MB (includes Bun runtime) - -The binaries are completely standalone and don't require Bun or Node.js to be installed. - ## Testing -The project includes comprehensive test coverage: - ```bash -# Run all tests -bun test - -# Run specific test files -bun test src/utils/paths.test.ts -bun test src/utils/config.test.ts -bun test src/db/queries.test.ts -bun test src/commands/commands.test.ts -bun test src/commands/index.test.ts -bun test src/converter/scip-parser.test.ts +bun test ./test/ # full suite +bun test test/tree-sitter/ # tree-sitter tests only +bun run type-check # tsc +bun run biome:format # format src/ and test/ ``` -Test coverage: - -- **Unit Tests**: Path utilities, config management, error handling -- **Integration Tests**: Database queries with example database -- **Command Tests**: CLI commands and initialization -- **Parser Tests**: SCIP protobuf parsing and conversion -- **Index Tests**: Database schema and denormalized fields - -## Debug Logging +Tests use real fixture data from `test/fixtures/`. The tree-sitter tests are pure unit tests — they mock `Parser.QueryCapture[]` objects and run against the actual parse functions without loading any wasm grammar. -The CLI uses the [`debug`](https://www.npmjs.com/package/debug) library for verbose logging during development and troubleshooting. Enable debug output using the `DEBUG` environment variable: +## Building ```bash -# Show all dora debug output -DEBUG=dora:* dora index +bun run build # standalone binary for current platform → dist/dora +bun run build:npm # bun-target JS bundle → dist/index.js (used by npm package) +bun run build:all # all platform binaries (linux-x64, darwin-x64, darwin-arm64, windows-x64) +``` -# Show only converter logs (useful for performance debugging) -DEBUG=dora:converter dora index +Binaries are ~57MB on macOS/Linux, ~58MB on Windows. They include the Bun runtime and have no external dependencies. -# Show only index command logs -DEBUG=dora:index dora index +## Debug logging -# Show multiple namespaces -DEBUG=dora:index,dora:converter dora index +```bash +DEBUG=dora:* dora index # all namespaces +DEBUG=dora:converter dora index # SCIP parsing and DB conversion +DEBUG=dora:index dora index # index command only +DEBUG=dora:db dora symbol Foo # database queries ``` -**Available namespaces:** - -- `dora:index` - Index command progress and timing -- `dora:converter` - SCIP parsing and database conversion details -- `dora:db` - Database operations and queries -- `dora:config` - Configuration loading and validation +Available namespaces: `dora:index`, `dora:converter`, `dora:db`, `dora:config`. -**Example output:** +Example output: -```bash +``` $ DEBUG=dora:* dora index dora:index Loading configuration... +0ms dora:index Config loaded: root=/path/to/project +2ms @@ -125,109 +63,74 @@ $ DEBUG=dora:* dora index dora:converter Processing files: 412/412 (100%) +265ms ``` -## Code Style - -- Use TypeScript for all source code -- Follow existing code formatting (we use Bun's default formatter) -- Write descriptive variable and function names -- Add JSDoc comments for public APIs -- Keep functions focused and under 50 lines when possible +## Code conventions -## Pull Request Process +- Single object parameter — never multiple positional params +- No inline comments, no section separators, no file headers +- No `any` — use `unknown` or proper types +- Boolean variables prefixed with `is` or `has` +- Use `type` not `interface` +- Output JSON to stdout, errors to stderr as `{"error": "message"}`, exit 1 on error -1. Fork the repository -2. Create a feature branch (`git checkout -b feature/my-feature`) -3. Make your changes with tests -4. Run the test suite (`bun test`) -5. Build the binaries (`bun run build:all`) -6. Commit your changes (`git commit -am 'Add my feature'`) -7. Push to the branch (`git push origin feature/my-feature`) -8. Create a Pull Request +## Adding a command -### PR Requirements +1. Create `src/commands/mycommand.ts` and export a function +2. Register it in `src/index.ts` +3. Add MCP tool definition in `src/mcp/metadata.ts` and handler in `src/mcp/handlers.ts` +4. Add tests in `test/commands/` -- All tests must pass -- New features should include tests -- Update documentation (README.md, CLAUDE.md) if needed -- Follow the existing code style -- Provide a clear description of the changes +## Adding a tree-sitter language -## Architecture +1. Create `src/tree-sitter/languages/mylang.ts` — export `functionQueryString`, `classQueryString`, `parseFunctionCaptures`, `parseClassCaptures` +2. Register it in `src/tree-sitter/languages/registry.ts` with its grammar name and file extensions +3. Add tests in `test/tree-sitter/mylang-captures.test.ts` — mock `Parser.QueryCapture[]` objects, test all capture variants, deduplication, and edge cases. See `test/tree-sitter/function-captures.test.ts` as the reference implementation. -dora is built around SCIP (Source Code Intelligence Protocol) indexes and SQLite for fast querying: +The grammar wasm file is resolved automatically from local `node_modules`, global bun packages, or an explicit path in `.dora/config.json` under `treeSitter.grammars.`. -### Key Components - -- **src/commands/** - CLI command implementations -- **src/converter/** - SCIP protobuf parser and SQLite converter -- **src/db/** - Database schema and queries -- **src/utils/** - Shared utilities (config, paths, errors) -- **src/types.ts** - TypeScript type definitions - -### Database Schema - -The database uses denormalized fields for performance: - -- **files** - File metadata with symbol/dependency counts -- **symbols** - Symbol definitions with location and kind -- **dependencies** - File-to-file dependencies with symbol lists -- **symbol_references** - Where symbols are used -- **packages** - External package information -- **metadata** - System metadata (last indexed, counts) - -For detailed schema and query patterns, see [CLAUDE.md](./CLAUDE.md). - -## Common Development Tasks - -### Adding a New Command - -1. Create a new file in `src/commands/` (e.g., `mynewcommand.ts`) -2. Implement the command function -3. Export the command in `src/index.ts` -4. Add tests in `src/commands/mynewcommand.test.ts` -5. Update README.md command reference - -### Adding a New Query +## Adding a query 1. Add the query function in `src/db/queries.ts` -2. Add tests in `src/db/queries.test.ts` -3. Use the query in your command +2. Add tests in `test/db/queries.test.ts` +3. Use it in your command -### Modifying the Database Schema +## Modifying the database schema 1. Update `src/converter/schema.sql` 2. Update conversion logic in `src/converter/convert.ts` -3. Increment the schema version if needed -4. Add migration logic if backward compatibility is required +3. Add migration logic if backward compatibility is required -## Troubleshooting Development Issues - -### Tests Failing +## Architecture -- Ensure you have the latest dependencies: `bun install` -- Check that `.dora/dora.db` exists: `dora index` -- Run tests in verbose mode: `DEBUG=* bun test` +``` +src/ +├── commands/ # one file per CLI command +├── converter/ # SCIP protobuf parser + SQLite converter +├── db/ # schema and all SQL queries +├── mcp/ # MCP server, tool definitions, handlers +├── schemas/ # Zod schemas and inferred types +├── tree-sitter/ # grammar discovery, parser, language implementations +└── utils/ # config, errors, output formatting +``` -### Build Issues +Key tables: `files`, `symbols`, `dependencies`, `symbol_references`, `packages`, `documents`, `metadata`. -- Clear the dist directory: `rm -rf dist` -- Reinstall Bun if needed -- Check Bun version: `bun --version` (should be 1.0+) +Denormalized counts (`symbol_count`, `dependency_count`, `dependent_count`, `reference_count`) are updated after every index run and make most queries O(1) lookups. -### Local Development +For detailed schema and query patterns, see [CLAUDE.md](./CLAUDE.md). -- Use `bun link` to link the development version -- Test with `dora --version` to ensure you're using the dev version -- Unlink with `bun unlink` when done +## Troubleshooting -## Questions? +**Tests failing:** run `bun install` to sync deps, ensure `.dora/dora.db` exists (`dora index`). -- Open an issue for bugs or feature requests -- Start a discussion for architecture questions -- Check [CLAUDE.md](./CLAUDE.md) for detailed implementation notes +**Build issues:** clear `dist/` and retry. Check `bun --version` is 1.0+. -> `dora` is intentionally minimal. Before proposing new features, consider if the problem can be solved with existing commands or `dora query` +**Local dev:** `bun link` points the `dora` binary at the source. `bun unlink` to restore. -## License +## Pull request checklist -By contributing, you agree that your contributions will be licensed under the MIT License. +- `bun test ./test/` passes +- `bun run type-check` passes +- `bun run biome:format` applied +- New commands have tests +- New tree-sitter languages have capture tests +- [CLAUDE.md](./CLAUDE.md) updated if architecture changed diff --git a/README.md b/README.md index f0b59d9..ed9dd02 100644 --- a/README.md +++ b/README.md @@ -1,584 +1,177 @@ -# dora - Code Context CLI for AI Agents +# dora -Stop wasting tokens on grep/find/glob. Give your AI agent fast, structured code intelligence. +A CLI that turns a SCIP index into a queryable SQLite database. Gives AI agents structured answers about your codebase instead of making them grep files and read imports. -## Features +## Why -- **Instant answers** - Pre-computed aggregates mean no waiting for grep/find/glob to finish or tokens wasted on file reads -- **Understand relationships** - See what depends on what without reading import statements or parsing code -- **Find issues fast** - Detect circular dependencies, coupling, and complexity hotspots with pre-indexed data -- **Track usage** - Know where every symbol is used across your codebase in milliseconds, not minutes -- **Language-agnostic** - Works with any SCIP-compatible indexer (TypeScript, Java, Rust, Python, etc.) +When an AI agent needs to understand code, it typically reads files, searches for patterns, and traces imports manually. This is slow, burns tokens, and doesn't scale past a few hundred files. -## See It In Action +dora pre-indexes your codebase into SQLite. Questions like "what depends on this file?", "where is this symbol used?", and "which files are most coupled?" become millisecond queries instead of multi-step explorations. -### Typical Workflow Without dora +## Setup -![Baseline CLI workflow showing grep/find approach](docs/public/baseline-cli.gif) +### Install -### With dora CLI - -![dora CLI workflow showing fast structured queries](docs/public/dora-cli.gif) - -## System Requirements - -- **Binary users**: No dependencies - standalone executable -- **From source**: Bun 1.0+ required -- **SCIP indexer**: Language-specific (e.g., scip-typescript for TS/JS) -- **Supported OS**: macOS, Linux, Windows -- **Disk space**: ~5-50MB for index (varies by codebase size) - -## Installation - -### Option 1: Download Pre-built Binary (Recommended) - -Download the latest binary for your platform from the [releases page](https://github.com/butttons/dora/releases): +Download the latest binary from the [releases page](https://github.com/butttons/dora/releases): ```bash -# macOS (ARM64) +# macOS (ARM) curl -L https://github.com/butttons/dora/releases/latest/download/dora-darwin-arm64 -o dora -chmod +x dora -sudo mv dora /usr/local/bin/ +chmod +x dora && sudo mv dora /usr/local/bin/ # macOS (Intel) curl -L https://github.com/butttons/dora/releases/latest/download/dora-darwin-x64 -o dora -chmod +x dora -sudo mv dora /usr/local/bin/ +chmod +x dora && sudo mv dora /usr/local/bin/ # Linux curl -L https://github.com/butttons/dora/releases/latest/download/dora-linux-x64 -o dora -chmod +x dora -sudo mv dora /usr/local/bin/ - -# Windows -# Download dora-windows-x64.exe and add to PATH +chmod +x dora && sudo mv dora /usr/local/bin/ ``` -### Option 2: Install via npm - -Requires [Bun](https://bun.sh) runtime installed. +Or via npm (requires [Bun](https://bun.sh)): ```bash bun install -g @butttons/dora ``` -Or run without installing: - -```bash -bunx @butttons/dora -``` - -### Option 3: Build from Source - -```bash -# Install Bun (if not already installed) -curl -fsSL https://bun.sh/install | bash - -# Clone the repository -git clone https://github.com/butttons/dora.git -cd dora - -# Install dependencies -bun install - -# Build the binary -bun run build - -# The binary will be at dist/dora -# Move it to your PATH -sudo mv dist/dora /usr/local/bin/ -``` - -### Install SCIP Indexer +### Install a SCIP indexer -You'll need a SCIP indexer for your language. For TypeScript/JavaScript: +dora needs a SCIP index to work. Install one for your language: ```bash -# Install scip-typescript globally +# TypeScript / JavaScript npm install -g @sourcegraph/scip-typescript - -# Verify installation -scip-typescript --help ``` -For other languages, see [SCIP Indexers](#scip-indexers). - -## AI Agent Integration - -**→ See [AGENTS.README.md](AGENTS.README.md) for complete integration guides** for: +Other languages: [scip-java](https://github.com/sourcegraph/scip-java), [rust-analyzer](https://github.com/rust-lang/rust-analyzer), [scip-python](https://github.com/sourcegraph/scip-python), [scip-ruby](https://github.com/sourcegraph/scip-ruby), [scip-clang](https://github.com/sourcegraph/scip-clang), [scip-dotnet](https://github.com/sourcegraph/scip-dotnet), [scip-dart](https://github.com/Workiva/scip-dart). -- **Claude Code** - Skills, hooks, auto-indexing -- **OpenCode** - Agent system integration -- **Cursor** - Custom commands and rules -- **Windsurf** - Skills, AGENTS.md, and rules -- **Other AI agents** - Generic integration using SKILL.md and SNIPPET.md - -Quick start for any agent: - -```bash -dora init && dora index # Initialize and index your codebase -dora cookbook show agent-setup --format markdown # Get setup instructions for your agent -dora status # Verify index is ready -``` - -## Claude Code Integration - -dora integrates with Claude Code via settings and optional skill configuration. Just add these files to your project: - -**1. Add to `.claude/settings.json`** (enables auto-indexing and permissions): - -```json -{ - "permissions": { - "allow": ["Bash(dora:*)", "Skill(dora)"] - }, - "hooks": { - "SessionStart": [ - { - "hooks": [ - { - "type": "command", - "command": "dora status 2>/dev/null && (dora index > /tmp/dora-index.log 2>&1 &) || echo 'dora not initialized. Run: dora init && dora index'" - } - ] - } - ], - "Stop": [ - { - "hooks": [ - { - "type": "command", - "command": "(dora index > /tmp/dora-index.log 2>&1 &) || true" - } - ] - } - ] - } -} -``` - -**2. (Optional) Add the dora skill** at `.claude/skills/dora/SKILL.md`: - -After running `dora init`, create a symlink: - -```bash -mkdir -p .claude/skills/dora -ln -s ../../../.dora/docs/SKILL.md .claude/skills/dora/SKILL.md -``` - -This enables the `/dora` command in Claude Code. [View the skill file](https://github.com/butttons/dora/blob/main/src/templates/docs/SKILL.md). - -**3. Add to CLAUDE.md** (after running `dora init`): - -```bash -cat .dora/docs/SNIPPET.md >> CLAUDE.md -``` - -This gives Claude quick access to dora commands and guidance on when to use dora for code exploration. The snippet includes command reference and best practices. - -**4. Initialize dora:** +### Initialize ```bash +cd your-project dora init dora index ``` -**What this gives you:** - -- Auto-indexing after each Claude turn -- Pre-approved permissions (no prompts for dora commands) -- Session startup checks -- CLAUDE.md context for better code exploration - -**Troubleshooting:** - -- **Index not updating?** Check `/tmp/dora-index.log` for errors -- **dora not found?** Ensure dora is in PATH: `which dora` - -## MCP Server +`dora init` creates `.dora/config.json`. `dora index` runs the SCIP indexer and converts the output to SQLite. -dora can run as an MCP (Model Context Protocol) server. +## Usage -### Quick Start +### Exploring a codebase ```bash -# Start MCP server (runs in foreground) -dora mcp +dora map # file count, symbol count, packages +dora ls src/ # files in a directory with symbol/dep counts +dora status # index health and grammar availability ``` -### Claude Code - -Add the MCP server with one command: +### Finding code ```bash -claude mcp add --transport stdio dora -- dora mcp +dora symbol AuthService # find symbols by name +dora file src/auth/service.ts # symbols, dependencies, dependents for a file +dora refs validateToken # all references to a symbol across the codebase ``` -### Other MCP Clients - -Add to your MCP client configuration: - -```json -{ - "mcpServers": { - "dora": { - "type": "stdio", - "command": "dora", - "args": ["mcp"] - } - } -} -``` - -### What You Get - -All dora commands are available as MCP tools: - -- `dora_status` - Check index health -- `dora_map` - Get codebase overview -- `dora_symbol` - Search for symbols -- `dora_file` - Analyze files with dependencies -- `dora_deps` / `dora_rdeps` - Explore dependencies -- And all other dora commands - -## Quick Start - -### 1. Initialize +### Understanding dependencies ```bash -dora init -``` - -This creates a `.dora/` directory with a default config. - -### 2. Configure Commands - -Edit `.dora/config.json` to configure your SCIP indexer: - -**For TypeScript/JavaScript:** - -```json -{ - "commands": { - "index": "scip-typescript index --output .dora/index.scip" - } -} +dora deps src/auth/service.ts --depth 2 # what this file imports +dora rdeps src/auth/service.ts --depth 2 # what imports this file +dora adventure src/a.ts src/b.ts # shortest path between two files ``` -**For Rust:** - -```json -{ - "commands": { - "index": "rust-analyzer scip . --output .dora/index.scip" - } -} -``` +### Tree-sitter analysis (TypeScript / JavaScript) -### 3. Index Your Codebase +These commands parse source directly without needing an index: ```bash -# If commands are configured: -dora index - -# Or manually: -scip-typescript index --output .dora/index.scip +dora fn src/auth/service.ts # functions with complexity, params, return type, LOC +dora class src/auth/service.ts # classes with methods, implements, decorators +dora smells src/auth/service.ts # high complexity, long functions, too many params, TODOs ``` -### 4. Try It Out +Install a grammar to enable these: ```bash -# Check index status -dora status - -# Get codebase overview -dora map - -# Find a symbol -dora symbol Logger +bun add -g tree-sitter-typescript ``` -### 5. Example Workflow +### Architecture ```bash -# Find a class definition -dora symbol AuthService - -# Explore the file -dora file src/auth/service.ts - -# See what depends on it -dora rdeps src/auth/service.ts --depth 2 - -# Check for circular dependencies -dora cycles +dora cycles # bidirectional dependencies (A imports B, B imports A) +dora coupling --threshold 5 # file pairs with high symbol sharing +dora complexity --sort complexity # files ranked by change risk +dora treasure # most imported files +dora lost # symbols with zero references ``` -### 6. Learn Custom Queries - -New to dora? The cookbook has recipes with real examples: +### Documentation ```bash -# Start here - complete walkthrough -dora cookbook show quickstart - -# Find class methods -dora cookbook show methods - -# Track symbol references -dora cookbook show references - -# Find exported APIs -dora cookbook show exports +dora docs # list indexed markdown/text files +dora docs search "authentication" # full-text search across docs +dora docs show docs/api.md # which symbols and files a doc references ``` -All recipes include tested SQL patterns from real codebases. - -## Commands Overview - -### Setup & Status +### Custom queries ```bash -dora init # Initialize in repo -dora index # Index codebase -dora status # Show index health -dora map # High-level statistics +dora schema # database schema +dora cookbook show quickstart # walkthrough with real SQL examples +dora cookbook show methods # query patterns for finding methods +dora query "SELECT path, symbol_count FROM files ORDER BY symbol_count DESC LIMIT 10" ``` -### Code Navigation +### MCP server ```bash -dora ls [directory] # List files in directory with metadata -dora symbol # Find symbols by name -dora file # File info with dependencies -dora refs # Find all references -dora deps --depth 2 # Show dependencies -dora rdeps --depth 2 # Show dependents -dora adventure # Find shortest path -``` - -### Documentation +dora mcp # start MCP server (stdio) -```bash -dora docs # List all documentation files -dora docs --type md # Filter by document type -dora docs search # Search documentation content -dora docs show # Show document details +# Claude Code +claude mcp add --transport stdio dora -- dora mcp ``` -### Architecture Analysis - -```bash -dora cycles # Find bidirectional dependencies -dora coupling --threshold 5 # Find tightly coupled files -dora complexity --sort complexity # High-impact files -dora treasure --limit 20 # Most referenced files -dora lost --limit 50 # Potentially dead code -dora leaves --max-dependents 3 # Leaf nodes -``` +### Output format -### Advanced Queries +All commands output [TOON](https://github.com/toon-format/toon) by default — a compact JSON encoding optimized for LLM token usage. Pass `--json` for standard JSON. ```bash -dora schema # Show database schema -dora cookbook show [recipe] # Show query pattern examples -dora query "" # Execute raw SQL (read-only) -dora changes # Changed/impacted files -dora exports # List exports -dora imports # Show imports -dora graph # Dependency graph +dora status --json ``` -## Command Reference - -Quick reference for all commands with common flags: - -### Setup Commands +## AI agent integration -| Command | Description | Common Flags | -| ------------- | ----------------------------- | --------------------------------------------- | -| `dora init` | Initialize dora in repository | - | -| `dora index` | Build/update index | `--full`, `--skip-scip`, `--ignore ` | -| `dora status` | Check index status | - | -| `dora map` | High-level statistics | - | - -### Code Navigation - -| Command | Description | Common Flags | -| ---------------------------- | ------------------------------ | ----------------------------- | -| `dora ls [directory]` | List files in directory | `--limit N`, `--sort ` | -| `dora file ` | Analyze file with dependencies | - | -| `dora symbol ` | Search for symbols | `--kind `, `--limit N` | -| `dora refs ` | Find all references | - | -| `dora deps ` | Show dependencies | `--depth N` (default: 1) | -| `dora rdeps ` | Show reverse dependencies | `--depth N` (default: 1) | -| `dora adventure ` | Find dependency path | - | - -### Documentation - -| Command | Description | Common Flags | -| -------------------------- | ---------------------------- | ---------------------------------- | -| `dora docs` | List all documentation files | `--type ` (md, txt) | -| `dora docs search ` | Search documentation content | `--limit N` (default: 20) | -| `dora docs show ` | Show document metadata | `--content` (include full content) | - -### Architecture Analysis - -| Command | Description | Common Flags | -| ----------------- | ------------------------------- | ---------------------------- | -| `dora cycles` | Find bidirectional dependencies | `--limit N` (default: 50) | -| `dora coupling` | Find tightly coupled files | `--threshold N` (default: 5) | -| `dora complexity` | Show complexity metrics | `--sort ` | -| `dora treasure` | Most referenced files | `--limit N` (default: 10) | -| `dora lost` | Find unused symbols | `--limit N` (default: 50) | -| `dora leaves` | Find leaf nodes | `--max-dependents N` | - -### Advanced Commands - -| Command | Description | Common Flags | -| ----------------------------- | --------------------------- | ------------------------------------------ | -| `dora schema` | Show database schema | - | -| `dora cookbook show [recipe]` | Query pattern cookbook | `quickstart`, `methods`, `refs`, `exports` | -| `dora query ""` | Execute raw SQL (read-only) | - | -| `dora changes ` | Git impact analysis | - | -| `dora exports ` | List exported symbols | - | -| `dora imports ` | Show file imports | - | -| `dora graph ` | Dependency graph | `--depth N`, `--direction` | - -## SCIP Indexers - -- [scip-typescript](https://github.com/sourcegraph/scip-typescript): TypeScript, JavaScript -- [scip-java](https://github.com/sourcegraph/scip-java): Java, Scala, Kotlin -- [rust-analyzer](https://github.com/rust-lang/rust-analyzer): Rust -- [scip-clang](https://github.com/sourcegraph/scip-clang): C++, C -- [scip-ruby](https://github.com/sourcegraph/scip-ruby): Ruby -- [scip-python](https://github.com/sourcegraph/scip-python): Python -- [scip-dotnet](https://github.com/sourcegraph/scip-dotnet): C#, Visual Basic -- [scip-dart](https://github.com/Workiva/scip-dart): Dart -- [scip-php](https://github.com/davidrjenni/scip-php): PHP - -## Output Format - -All commands output [TOON](https://github.com/toon-format/toon) (Token-Oriented Object Notation) by default. TOON is a compact, human-readable encoding of JSON that minimizes tokens for LLM consumption. Pass `--json` to any command for JSON output. +For agent-specific setup (hooks, skills, AGENTS.md snippets) for Claude Code, OpenCode, Cursor, and Windsurf: ```bash -# Default: TOON output -dora status - -# JSON output -dora --json status -dora status --json +dora cookbook show agent-setup ``` -Errors always go to stderr as JSON with exit code 1. - -### TOON vs JSON size comparison - -Measured on dora's own codebase (79 files, 3167 symbols): - -| Command | JSON | TOON | Savings | -|---|---|---|---| -| `status` | 206 B | 176 B | 15% | -| `map` | 68 B | 62 B | 9% | -| `ls src/commands` | 2,258 B | 975 B | **57%** | -| `ls` (all files) | 6,324 B | 2,644 B | **58%** | -| `file src/index.ts` | 6,486 B | 6,799 B | -5% | -| `symbol setupCommand` | 130 B | 130 B | 0% | -| `refs wrapCommand` | 510 B | 549 B | -8% | -| `deps (depth 2)` | 2,158 B | 1,332 B | **38%** | -| `rdeps (depth 2)` | 1,254 B | 802 B | **36%** | -| `adventure` | 110 B | 97 B | 12% | -| `leaves` | 142 B | 129 B | 9% | -| `exports` | 488 B | 511 B | -5% | -| `imports` | 1,978 B | 1,998 B | -1% | -| `lost` | 1,876 B | 1,987 B | -6% | -| `treasure` | 893 B | 577 B | **35%** | -| `cycles` | 14 B | 11 B | 21% | -| `coupling` | 35 B | 31 B | 11% | -| `complexity` | 2,716 B | 932 B | **66%** | -| `schema` | 6,267 B | 4,389 B | **30%** | -| `query` | 692 B | 464 B | **33%** | -| `docs` | 1,840 B | 745 B | **60%** | -| `docs search` | 277 B | 171 B | **38%** | -| `docs show` | 820 B | 870 B | -6% | -| `graph` | 2,434 B | 1,894 B | **22%** | -| `changes` | 1,112 B | 1,026 B | 8% | - -Commands with uniform arrays of objects (ls, complexity, docs, treasure) see 35-66% reduction. Nested or non-uniform outputs (file, refs, exports) are roughly equal or slightly larger. - -## Debug Logging - -For debug logging, testing, building, and development instructions, see [CONTRIBUTING.md](./CONTRIBUTING.md). - -## Troubleshooting - -### Common Issues +Or see [AGENTS.README.md](./AGENTS.README.md). -| Issue | Solution | -| -------------------------- | ------------------------------------------------------------ | -| **Database not found** | Run `dora index` to create the database | -| **File not in index** | Check if file is in .gitignore, run `dora index` | -| **Stale results** | Run `dora index` to rebuild | -| **Slow queries** | Use `--depth 1` when possible, reduce `--limit` | -| **Symbol not found** | Ensure index is up to date: `dora status`, then `dora index` | -| **dora command not found** | Ensure dora is in PATH: `which dora`, reinstall if needed | +## How it works -### Integration Issues +dora has two layers: -**Claude Code index not updating:** +**SCIP layer** — runs your configured indexer (e.g. `scip-typescript`) to produce a `.scip` protobuf file, then parses it and loads it into SQLite. This gives you symbol definitions, references, and file-to-file dependencies derived from actual import resolution. -- Check `/tmp/dora-index.log` for errors -- Verify dora is in PATH: `which dora` -- Test manually: `dora index` -- Ensure `dora index` is in the `allow` permissions list in `.claude/settings.json` +**Tree-sitter layer** — parses source files directly using WebAssembly grammars. This runs on-demand per file and extracts things SCIP doesn't cover: function signatures, cyclomatic complexity, class hierarchy details, and code smells. -**Stop hook not firing:** +The SQLite schema uses denormalized counts (`symbol_count`, `dependency_count`, `dependent_count`, `reference_count`) so most queries are index lookups rather than aggregations. -- Verify `.claude/settings.json` syntax is correct (valid JSON) -- Check that the hook runs by viewing verbose logs -- Try manually running the hook command - -**Want to see indexing progress:** - -- Edit `.claude/settings.json` Stop hook -- Change command to: `"DEBUG=dora:* dora index 2>&1 || true"` (removes background `&`) -- You'll see progress after each turn, but will wait 15-30s - -### Performance Issues - -**Index takes too long:** - -- Run SCIP indexer separately if it supports caching -- Use background indexing mode in Claude Code integration -- Check if your SCIP indexer can be optimized - -**Queries are slow:** - -- Use `--depth 1` instead of deep traversals -- Reduce `--limit` for large result sets -- Ensure database indexes are created (automatic) -- Run `dora index` if database is corrupted +``` +.dora/ +├── config.json # indexer command, ignore patterns, grammar paths +├── index.scip # raw SCIP protobuf output +└── dora.db # SQLite database (the thing dora actually queries) +``` ## Contributing -Contributions are welcome! For development setup, testing, building binaries, and code style guidelines, see [CONTRIBUTING.md](./CONTRIBUTING.md). - -Quick start: - -1. Fork the repository -2. Create a feature branch -3. Make your changes with tests (`bun test`) -4. Submit a pull request - -For detailed architecture and development guidelines, see [CLAUDE.md](./CLAUDE.md). +See [CONTRIBUTING.md](./CONTRIBUTING.md). ## License MIT - -## Links - -- **AI Agent Integration**: [AGENTS.README.md](./AGENTS.README.md) - Integration guides for Claude Code, OpenCode, Cursor, Windsurf -- **GitHub**: [https://github.com/butttons/dora](https://github.com/butttons/dora) -- **SCIP Protocol**: [https://github.com/sourcegraph/scip](https://github.com/sourcegraph/scip) -- **Claude Code**: [https://claude.ai/code](https://claude.ai/code) diff --git a/bun.lock b/bun.lock index 3e1a19a..7900dd7 100644 --- a/bun.lock +++ b/bun.lock @@ -13,6 +13,7 @@ "debug": "^4.4.3", "ignore": "^5.3.0", "ts-pattern": "^5.9.0", + "web-tree-sitter": "^0.26.6", "zod": "^4.3.5", }, "devDependencies": { @@ -402,6 +403,8 @@ "vary": ["vary@1.1.2", "", {}, "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg=="], + "web-tree-sitter": ["web-tree-sitter@0.26.6", "", {}, "sha512-fSPR7VBW/fZQdUSp/bXTDLT+i/9dwtbnqgEBMzowrM4U3DzeCwDbY3MKo0584uQxID4m/1xpLflrlT/rLIRPew=="], + "which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="], "wrappy": ["wrappy@1.0.2", "", {}, "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ=="], diff --git a/docs/AGENTS.md b/docs/AGENTS.md new file mode 100644 index 0000000..b2a5969 --- /dev/null +++ b/docs/AGENTS.md @@ -0,0 +1,42 @@ +# docs/AGENTS.md + +Documentation website for dora CLI at https://dora-cli.dev. Built with Astro, deployed to Cloudflare Workers. + +**For dora CLI context, commands, and schema, see `../AGENTS.md`.** + +## Stack + +- Framework: Astro 5.x +- Styling: Tailwind CSS 4.x via `@tailwindcss/vite` +- Icons: lucide-astro +- Deployment: Cloudflare Workers via `@astrojs/cloudflare` + +## Structure + +``` +src/ +├── pages/ +│ ├── index.astro # Landing page +│ ├── docs.astro # Full documentation +│ ├── commands.astro # Command reference +│ └── og-image.astro # OG image (SSR) +├── components/ # Shared components +└── layouts/ + └── Layout.astro # Base layout, nav, footer +``` + +## Dev + +```bash +bun run dev # http://localhost:4321 +bun run build # production build → dist/ +bun run deploy # deploy to Cloudflare Workers +``` + +## Keeping content in sync + +When adding or changing dora CLI commands, update `commands.astro` to match. The command reference should mirror what `dora --help` outputs. + +## Styling + +Dark theme: `zinc-950` page background, `zinc-900` cards, `zinc-800` borders. Primary: `blue-400/500`. Body text: `zinc-300`. diff --git a/docs/CLAUDE.md b/docs/CLAUDE.md deleted file mode 100644 index b09174f..0000000 --- a/docs/CLAUDE.md +++ /dev/null @@ -1,550 +0,0 @@ -# docs/CLAUDE.md - -This file provides guidance for maintaining and updating the dora CLI documentation website. - -**Parent Context:** For dora CLI tool context, commands, and database schema, see `../CLAUDE.md` in the project root. - -## Purpose - -The `docs/` directory contains the documentation website for dora CLI at https://dora-cli.dev. This is a static site built with Astro that provides: - -- Landing page with quick start instructions -- Comprehensive documentation for users -- Command reference guide -- Architecture and design philosophy -- AI agent integration examples - -## Tech Stack - -- **Framework:** Astro 5.x (static site generator) -- **Styling:** Tailwind CSS 4.x with `@tailwindcss/vite` plugin -- **Icons:** lucide-astro (Terminal, Zap, Database, etc.) -- **Font:** DM Sans (loaded from Google Fonts) -- **Deployment:** Cloudflare Workers via @astrojs/cloudflare adapter -- **Build Tool:** Bun (package manager and runtime) - -## Project Structure - -``` -docs/ -├── src/ -│ ├── pages/ -│ │ ├── index.astro # Landing page (hero, features, quick start) -│ │ ├── docs.astro # Full documentation page -│ │ ├── commands.astro # Command reference -│ │ ├── architecture.astro # Design philosophy & use cases -│ │ └── og-image.astro # Open Graph image generator -│ └── layouts/ -│ └── Layout.astro # Base layout (head tags, nav, footer) -├── public/ # Static assets -├── package.json -├── tsconfig.json -├── tailwind.config.js # Tailwind CSS 4.x config -└── astro.config.mjs # Astro + Cloudflare config -``` - -## Pages Overview - -### 1. Landing Page (`src/pages/index.astro`) - -The main entry point with several key sections: - -**Hero Section:** -- Large "dora" title with Terminal icon -- Tagline: "the explorer for your codebase" -- Two CTAs: "Quick Start" and "Documentation" - -**AI Agent Comparison:** -- Side-by-side comparison of typical AI workflows vs. dora commands -- Examples: finding classes, analyzing dependencies, understanding architecture -- Uses monospace font and color-coded boxes (zinc-900 vs. blue-900) - -**Feature Cards:** -- Grid layout with icon, title, and description -- Features: Fast queries, dependency tracking, symbol search, etc. -- Icons from lucide-astro (Database, Network, Search, etc.) - -**Quick Start Section:** -- Dynamic platform detection (macOS/Linux/Windows) -- Language selector for SCIP indexer installation (9 languages) -- Installation command generation based on selections -- Step-by-step workflow examples - -**Platform/Language Detection Logic:** -The landing page includes client-side JavaScript for dynamic installation instructions: - -```javascript -// Platform detection (macOS, Linux, Windows) -const platform = navigator.platform.includes('Mac') ? 'macos' - : navigator.platform.includes('Win') ? 'windows' - : 'linux'; - -// Language selector (TypeScript, JavaScript, Go, etc.) -const languages = ['typescript', 'javascript', 'go', 'rust', 'java', - 'python', 'ruby', 'c', 'cpp']; -``` - -### 2. Documentation Page (`src/pages/docs.astro`) - -Comprehensive user guide with sections: - -- **What is dora?** - Overview and purpose -- **Installation** - Platform-specific instructions -- **Quick Start** - Initialize, index, basic queries -- **Core Concepts** - SCIP indexes, symbols, dependencies -- **Command Overview** - Brief description of all command categories -- **AI Agent Integration** - How AI agents should use dora -- **Common Workflows** - Real-world usage patterns -- **Troubleshooting** - FAQ and common issues - -**Styling Pattern:** -- Prose sections with `text-zinc-300` for readability -- Code blocks with `bg-zinc-900 border-zinc-800` background -- Headers with gradient text (`text-blue-400`) -- Links with `text-blue-400 hover:text-blue-300` transitions - -### 3. Command Reference (`src/pages/commands.astro`) - -Organized into 5 command categories: - -1. **Setup & Status** - init, index, status, map, ls -2. **Query Commands** - file, symbol, refs, deps, rdeps, adventure -3. **Analysis Commands** - cycles, coupling, complexity, treasure, lost, leaves -4. **Graph & Export** - graph, exports, imports -5. **Advanced** - schema, query, changes - -**Command Card Structure:** -```astro -
-

- dora command [args] -

-

Description of what it does

-
- Example usage -
-
-``` - -### 4. Architecture Page (`src/pages/architecture.astro`) - -Explains the design philosophy and technical implementation: - -**Sections:** -- **Design Philosophy** - Why dora exists, goals, principles -- **How It Works** - SCIP indexing, protobuf parsing, SQLite storage -- **Performance** - Denormalized fields, incremental indexing -- **Use Cases** - AI agents, code exploration, refactoring, onboarding - -**Visual Hierarchy:** -- Large section headers with bottom borders -- Subsections with icon bullets -- Code examples with syntax highlighting -- Callout boxes for important notes - -### 5. OG Image (`src/pages/og-image.astro`) - -Generates Open Graph images for social media sharing: -- Simple text-based design with "dora" title -- Blue gradient background matching brand colors -- Used in meta tags for link previews - -## Styling Guidelines - -### Color Scheme - -**Primary Colors:** -- Blue: `blue-500` (primary actions), `blue-400` (text/links), `blue-300` (hover) -- Background: `zinc-950` (page bg), `zinc-900` (cards), `zinc-800` (borders) -- Text: `zinc-300` (body), `zinc-400` (muted), `zinc-500` (disabled) - -**Usage:** -```astro - - -``` - -**Feature Card:** -```astro -
- -

Fast Queries

-

Description text

-
-``` - -**Code Comparison (Before/After):** -```astro -
-
-
# Without dora
-
grep -rn "class Logger"
-
-
-
# With dora
-
dora symbol Logger
-
-
-``` - -## Dynamic Features - -### Platform Detection - -The landing page detects the user's OS to show appropriate installation instructions: - -**Implementation:** -```javascript -// In index.astro