Skip to content

chore: Infrastructure config, documentation & metadata updates #95

@mfittko

Description

@mfittko

Context

Spike PR: #88 (commit 7 of 7)
Merge order: 7/7 — Depends on all above; can also be split and merged alongside relevant issues.
Reference branch: spike/validate-docker-compose-stack — commit d4b0ec8


Overview

Update infrastructure configuration (Docker Compose, Helm, .env.example) for OpenAI-by-default architecture, comprehensively update all documentation to reflect new features, and refresh skill/plugin metadata.


Detailed Specification

1. Docker Compose (docker-compose.yml)

Service changes:

  • ollama: Moved to profiles: ["ollama"] — only starts with docker compose --profile ollama up
  • api: No longer depends on ollama; uses OpenAI embeddings by default
    • EMBED_PROVIDER: "openai", OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_EMBEDDING_MODEL from env
  • worker: EXTRACTOR_PROVIDER: "openai", concurrency increased from default to 8
    • EXTRACTOR_MAX_OUTPUT_TOKENS, OPENAI_API_KEY, OPENAI_BASE_URL from env

2. Environment Example (.env.example)

Port changes: 5432→26532 (Postgres), 8080→39180 (API) — avoid conflicts with common local services.

New variables documented:

Variable Default Purpose
RAGED_URL http://localhost:39180 CLI default API URL
EMBED_PROVIDER openai Embedding provider selection
OPENAI_API_KEY Required for OpenAI embeddings + worker
OPENAI_BASE_URL https://api.openai.com/v1 Custom endpoint
OPENAI_EMBEDDING_MODEL text-embedding-3-small Embedding model
EXTRACTOR_PROVIDER openai Worker extraction provider
EXTRACTOR_MAX_OUTPUT_TOKENS 16384 Max output tokens

Worker model defaults: All changed to gpt-4.1-mini.

3. Helm Chart

chart/values.yaml:

  • Worker extractorProvider changed to "auto"
  • Added openaiBaseUrl configurable field

chart/templates/worker-configmap.yaml:

  • Added OPENAI_BASE_URL env var injection

4. Documentation Updates

docs/02-local-dev.md

  • Simplified prerequisites: no Ollama model pulling needed
  • Base stack: docker compose up -d (Ollama not included)
  • Optional Ollama: docker compose --profile ollama up -d
  • Updated service table (API port 39180)
  • OpenAI-by-default guidance

docs/03-cli.md

  • Comprehensive query command documentation:
    • --minScore, --summary [level], --keywords, --unique
    • --full, --stdout, --download, --open
    • --collections, --allCollections
    • Positional query text support
  • New raged collections command section
  • Expanded raged enrich with --filter, --clear, --stats
  • Directory ingest metadata documentation
  • Many new examples with flag combinations

docs/05-helm-remote.md

  • Worker extractor provider: auto mode guidance
  • OpenAI-compatible endpoint configuration
  • New extractor config options documented

docs/06-troubleshooting.md

  • New error type: no_extractable_text explanation and remediation

docs/09-api-reference.md

  • POST /query documents minScore parameter with adaptive defaults table
  • POST /query/download-first — full endpoint documentation
  • POST /query/fulltext-first — full endpoint documentation
  • GET /collections — response format and field descriptions
  • POST /enrichment/clear — request/response documentation
  • POST /enrichment/enqueue — filter parameter
  • GET /enrichment/stats — collection and filter query parameters

5. Skill/Plugin Updates

skill/.claude-plugin/plugin.json

  • Updated keywords: Qdrant → pgvector, added OpenAI, Anthropic

skill/README.md

  • Custom port examples updated
  • OpenAI embedding configuration guidance
  • Removed Ollama model-pulling requirement

skill/SKILL.md

  • Updated skill description and metadata

skill/skills/raged/SKILL.md

  • Complete rewrite: CLI-first approach replacing curl-based examples
  • Description: "CLI-first workflow for raged semantic search"
  • Requires raged binary instead of curl + jq
  • Uses RAGED_URL and RAGED_API_TOKEN env vars
  • Query flag guide with recommendations for --collection, --collections, --allCollections
  • TopK guidance: 5-10 for Q&A, 20-50 for audits, 100 for inventory
  • Summary level recommendations
  • Ingestion, enrichment, graph sections rewritten for CLI
  • API fallback section for edge cases
  • Simplified troubleshooting table

skill/references/REFERENCE.md

  • Updated references for new architecture

6. AGENTS.md Updates

Root AGENTS.md

  • SRP principle references embeddings.ts instead of ollama.ts

api/AGENTS.md

  • Documents new env vars: EMBED_PROVIDER, OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_EMBEDDING_MODEL
  • Clarifies embeddings module orchestrates provider-specific calls
  • Removes duplicate BODY_LIMIT_BYTES from config table

7. New Files

  • .github/skills/ directory: Symlink to skill/skills/ (enables GitHub-native skill discovery)
  • skill/scripts/extract-chrome-bookmarks.mjs: 225-line utility to export Chrome bookmarks for ingest

Files (19 files)

File Status Purpose
docker-compose.yml Modified Ollama optional, OpenAI default
.env.example Modified New ports, env vars, defaults
chart/templates/worker-configmap.yaml Modified OPENAI_BASE_URL injection
chart/values.yaml Modified Auto provider, openaiBaseUrl
docs/00-vision.md Modified Minor update
docs/02-local-dev.md Modified Simplified, Ollama optional
docs/03-cli.md Modified Comprehensive CLI docs expansion
docs/05-helm-remote.md Modified Auto provider, OpenAI guidance
docs/06-troubleshooting.md Modified no_extractable_text error
docs/09-api-reference.md Modified New endpoints documented
skill/.claude-plugin/plugin.json Modified Keywords update
skill/README.md Modified OpenAI config guidance
skill/SKILL.md Modified Updated description
skill/references/REFERENCE.md Modified Updated references
skill/skills/raged/SKILL.md Modified CLI-first rewrite
.github/skills New Symlink for skill discovery
skill/scripts/extract-chrome-bookmarks.mjs New Chrome bookmark export utility
AGENTS.md Modified Reference embeddings.ts
api/AGENTS.md Modified Document new env vars

Acceptance Criteria

  • docker compose up -d works without Ollama (OpenAI-by-default)
  • docker compose --profile ollama up -d brings up Ollama stack
  • .env.example documents all current env vars correctly
  • Helm chart deploys with auto provider mode
  • docs/02-local-dev.md reflects simplified prerequisites
  • docs/03-cli.md covers all new CLI flags and commands
  • docs/09-api-reference.md documents all new endpoints
  • Skill SKILL.md uses CLI-first examples throughout
  • AGENTS.md references correct modules and env vars
  • .github/skills symlink works correctly

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions