Axon

Rust-based crawl, scrape, ingest, embed, query, and RAG engine with a unified CLI, MCP server, async workers, and web UI. Self-hosted research and knowledge-base backbone.

Version: 0.35.0 | License: MIT

Overview
Installation
CLI Reference
- Global Flags
- scrape
- crawl
- map
- extract
- search
- research
- embed
- query
- retrieve
- ask
- evaluate
- suggest
- ingest
- sessions
- refresh
- watch
- graph
- sources
- domains
- stats
- status
- doctor
- debug
- export
- artifacts
- migrate
- serve
- mcp
- Job Lifecycle Subcommands
MCP Server
Configuration
- Environment Variables
Infrastructure
- Docker Compose Stack
- Lite Mode
Ingest Sources
- GitHub
- Reddit
- YouTube
- AI Sessions
Vector Storage and RAG
Graph Operations
Browser Rendering
Performance Profiles
Web UI
Security
Development
End-to-End Workflow Example
Related Files

Overview

Axon is three things simultaneously:

A CLI binary: axon
An MCP server: axon mcp
A local stack supervisor: axon serve

The stack has the following components:

Crate / Directory	Role
`main.rs`, `lib.rs`	Binary entry and command dispatch
`crates/cli/`	Command handlers
`crates/core/`	Config, HTTP safety, content transforms
`crates/crawl/`	Crawl engine, render modes, sitemap backfill
`crates/jobs/`	Async job runtime, AMQP workers, state machine
`crates/vector/`	Qdrant operations, TEI embedding, RAG
`crates/ingest/`	GitHub, Reddit, YouTube, AI session ingestion
`crates/mcp/`	MCP server, tool schema, action router
`crates/services/`	Typed service entry points for CLI, MCP, and web
`apps/web/`	Next.js dashboard at port 49010
`docker/`, compose files	Infrastructure and runtime deployment

Installation

Prerequisites

Rust toolchain (see rust-toolchain.toml, currently stable 1.94.0+)
Docker Engine + Compose v2 plugin
GPU with NVIDIA CUDA (required for TEI; CPU fallback possible but slower)

Bootstrap

Run the setup script once:

./scripts/dev-setup.sh

This script:

Installs rustup and the pinned toolchain
Installs just, lefthook, cargo-nextest, cargo-watch, sccache
Checks Node.js ≥ v24 and pnpm ≥ v10
Runs pnpm install for apps/web
Creates .env from .env.example and auto-generates secrets
Starts production infrastructure containers

Optional flags:

./scripts/dev-setup.sh --build       # also compile the release binary
./scripts/dev-setup.sh --no-docker   # skip Docker startup

After the script, edit .env to set TEI_URL, OPENAI_*, and TAVILY_API_KEY.

Build

cargo build --release --bin axon
# or, with just:
just build

Install to PATH

just install
# symlinks target/release/axon to ~/.local/bin/axon

CLI Reference

All commands share global flags documented below. Commands listed as async by default enqueue a job and return immediately; pass --wait true to block until completion.

Global Flags

Core Behavior

Flag	Type	Default	Description
`--wait <bool>`	bool	`false`	Block until async job completes. Without this, async commands return a job ID immediately.
`--yes`	flag	—	Skip destructive confirmation prompts.
`--json`	flag	—	Machine-readable JSON output on stdout.
`--graph`	flag	—	Enable graph-enhanced retrieval for `ask` (requires Neo4j).
`--lite`	flag	—	Run in lite mode (SQLite, no external queue services).

Crawl and Scrape

Flag	Type	Default	Description
`--max-pages <n>`	u32	`0`	Page cap for crawl (0 = uncapped).
`--max-depth <n>`	usize	`5`	Maximum crawl depth from start URL.
`--render-mode <mode>`	enum	`auto-switch`	`http`, `chrome`, or `auto-switch`. Auto-switch tries HTTP first and falls back to Chrome when >60% of pages are thin.
`--format <fmt>`	enum	`markdown`	Output format: `markdown`, `html`, `rawHtml`, `json`.
`--include-subdomains <bool>`	bool	`false`	Crawl all subdomains of the start URL's parent domain.
`--respect-robots <bool>`	bool	`false`	Respect `robots.txt` directives.
`--discover-sitemaps <bool>`	bool	`true`	Discover and backfill URLs from sitemap.xml after crawl.
`--max-sitemaps <n>`	usize	`512`	Maximum sitemap URLs to backfill per crawl.
`--sitemap-since-days <n>`	u32	`0`	Only backfill sitemap URLs with `<lastmod>` within the last N days (0 = no filter).
`--min-markdown-chars <n>`	usize	`200`	Minimum markdown character count; pages below this are flagged thin.
`--drop-thin-markdown <bool>`	bool	`true`	Skip thin pages — do not save or embed them.
`--delay-ms <ms>`	u64	`0`	Delay between requests in milliseconds.
`--header <HEADER>`	string	—	Custom HTTP header in `Key: Value` format. Repeatable.

Output

Flag	Type	Default	Description
`--output-dir <dir>`	path	`.cache/axon-rust/output`	Directory for saved output files.
`--output <path>`	path	—	Explicit output file path (single-file commands).

Vector and Embedding

Flag	Type	Default	Description
`--collection <name>`	string	`cortex`	Qdrant collection name. Also set via `AXON_COLLECTION`.
`--embed <bool>`	bool	`true`	Auto-embed scraped content into Qdrant.
`--limit <n>`	usize	`10`	Result limit for search and query commands.
`--query <text>`	string	—	Query text (alternative to positional argument).
`--urls <csv>`	string	—	Comma-separated URL list (alternative to positional arguments).

Performance Tuning

Flag	Type	Default	Description
`--performance-profile <p>`	enum	`high-stable`	`high-stable`, `extreme`, `balanced`, `max`. Sets concurrency, timeout, and retry defaults.
`--batch-concurrency <n>`	usize	`16`	Concurrent connections for batch operations (1–512).
`--concurrency-limit <n>`	usize	—	Override crawl, sitemap, and backfill concurrency at once.
`--crawl-concurrency-limit <n>`	usize	profile	Override crawl concurrency.
`--sitemap-concurrency-limit <n>`	usize	profile	Override sitemap backfill concurrency.
`--backfill-concurrency-limit <n>`	usize	profile	Override backfill concurrency.
`--request-timeout-ms <ms>`	u64	profile	Per-request timeout in milliseconds.
`--fetch-retries <n>`	usize	profile	Number of retries on failed fetches.
`--retry-backoff-ms <ms>`	u64	profile	Backoff between retries in milliseconds.

Service URL Overrides

Flag	Env Var	Default
`--pg-url <url>`	`AXON_PG_URL`	`postgresql://axon:postgres@127.0.0.1:53432/axon`
`--redis-url <url>`	`AXON_REDIS_URL`	`redis://127.0.0.1:53379`
`--amqp-url <url>`	`AXON_AMQP_URL`	`amqp://axon:axonrabbit@127.0.0.1:45535/%2f`
`--qdrant-url <url>`	`QDRANT_URL`	`http://127.0.0.1:53333`
`--tei-url <url>`	`TEI_URL`	—
`--openai-base-url <url>`	`OPENAI_BASE_URL`	—
`--openai-api-key <key>`	`OPENAI_API_KEY`	—
`--openai-model <name>`	`OPENAI_MODEL`	—

Queue Configuration

Flag	Env Var	Default
`--crawl-queue <name>`	`AXON_CRAWL_QUEUE`	`axon.crawl.jobs`
`--extract-queue <name>`	`AXON_EXTRACT_QUEUE`	`axon.extract.jobs`
`--embed-queue <name>`	`AXON_EMBED_QUEUE`	`axon.embed.jobs`

scrape

Scrape one or more URLs. Runs inline (no queue). Validates URLs, fetches content, converts to the requested format, and optionally embeds into Qdrant.

axon scrape <url>... [FLAGS]
axon scrape --urls "<url1>,<url2>" [FLAGS]
axon scrape --url-glob "https://docs.example.com/{1..10}" [FLAGS]

Flag	Default	Description
`--format <fmt>`	`markdown`	Output format: `markdown`, `html`, `rawHtml`, `json`.
`--render-mode <mode>`	`auto-switch`	`http`, `chrome`, `auto-switch` (behaves like HTTP for scrape).
`--embed <bool>`	`true`	Batch-embed scraped markdown into Qdrant after all URLs finish.
`--output <path>`	—	Write output to a file (single URL only).
`--header "Key: Value"`	—	Repeatable custom HTTP headers.

Examples:

axon scrape https://example.com
axon scrape --urls "https://a.dev,https://b.dev"
axon scrape https://example.com --format html --output page.html
axon scrape https://example.com --embed false --json

crawl

Full site crawl. Async by default (enqueue and return). Use --wait true for synchronous mode.

axon crawl <url>... [FLAGS]
axon crawl --urls "<url1>,<url2>" [FLAGS]
axon crawl <SUBCOMMAND> [ARGS]

Key flags:

Flag	Default	Description
`--wait <bool>`	`false`	`false`: enqueue and return job IDs. `true`: run inline and block.
`--max-pages <n>`	`0`	Page cap (0 = uncapped).
`--max-depth <n>`	`5`	Maximum crawl depth.
`--render-mode <mode>`	`auto-switch`	`http`, `chrome`, `auto-switch`.
`--embed <bool>`	`true`	Queue embed job from crawl output.
`--sitemap-only`	`false`	Sync-only: run sitemap backfill without full crawl.

Job subcommands:

axon crawl status <job_id>
axon crawl cancel <job_id>
axon crawl errors <job_id>
axon crawl list
axon crawl cleanup
axon crawl clear
axon crawl recover
axon crawl worker
axon crawl audit <url>
axon crawl diff

Examples:

axon crawl https://example.com                         # async enqueue
axon crawl https://example.com --wait true             # sync
axon crawl https://example.com --render-mode chrome --max-pages 200
axon crawl status 550e8400-e29b-41d4-a716-446655440000

map

Discover all URLs from a site without scraping content. Combines crawler traversal with sitemap.xml discovery.

axon map <url> [FLAGS]

Returns a deduplicated, sorted URL list. No output files written.

extract

LLM-powered structured data extraction from URLs. Async by default.

axon extract <url>... [FLAGS]
axon extract <SUBCOMMAND> [ARGS]

Job subcommands follow the same pattern as crawl: status, cancel, errors, list, cleanup, clear, recover, worker.

search

Web search via Tavily. Automatically queues crawl jobs for top results.

axon search "<query>" [FLAGS]

Requires TAVILY_API_KEY. Results are de-duplicated by domain and optionally crawled.

research

Web research via Tavily AI search with LLM synthesis. Returns a structured research report.

axon research "<query>" [FLAGS]

Flag	Default	Description
`--limit <n>`	`10`	Number of Tavily results.
`--search-time-range`	—	Filter results by recency: `day`, `week`, `month`, `year`.

Requires TAVILY_API_KEY and AXON_ACP_ADAPTER_CMD.

embed

Embed a file, directory, or URL into Qdrant. Async by default.

axon embed [input] [FLAGS]
axon embed <SUBCOMMAND> [ARGS]

Input can be a file path, directory path, or URL. When omitted, reads from standard input.

Job subcommands: status, cancel, errors, list, cleanup, clear, recover, worker.

query

Semantic vector search against the Qdrant collection.

axon query "<text>" [FLAGS]

Flag	Default	Description
`--collection <name>`	`cortex`	Qdrant collection to search.
`--limit <n>`	`10`	Number of results.
`--since <date>`	—	Filter by index date (ISO 8601).
`--before <date>`	—	Filter by index date upper bound.

retrieve

Fetch stored document chunks from Qdrant by URL.

axon retrieve <url> [FLAGS]

Flag	Default	Description
`--max-points <n>`	—	Maximum chunks to return.
`--collection <name>`	`cortex`	Qdrant collection.

ask

RAG-powered Q&A. Runs synchronously. Retrieves relevant chunks, reranks them, builds a context window, and calls the configured LLM.

axon ask "<question>" [FLAGS]
axon ask --query "<question>" [FLAGS]

Flag	Default	Description
`--collection <name>`	`cortex`	Qdrant collection to search.
`--graph`	`false`	Enable graph-enhanced retrieval via Neo4j (requires `AXON_NEO4J_URL`).
`--diagnostics`	`false`	Print retrieval diagnostics.

RAG pipeline:

Embed the question via TEI
Query Qdrant for top AXON_ASK_CANDIDATE_LIMIT (default: 64) candidates
Filter by AXON_ASK_MIN_RELEVANCE_SCORE (default: 0.45)
Rerank by score; take top AXON_ASK_CHUNK_LIMIT (default: 10)
Backfill additional chunks from top AXON_ASK_FULL_DOCS (default: 4) documents
Assemble context up to AXON_ASK_MAX_CONTEXT_CHARS (default: 120,000) characters
Optionally inject Neo4j graph context block (when --graph is set)
Call the LLM via ACP adapter
Apply citation-quality gates and return the answer

axon ask "how does spider.rs handle JavaScript-heavy sites?"
axon ask "list all indexed rust crates" --collection rust-libs
axon ask "qdrant HNSW parameters" --diagnostics
axon ask "what is the default chunk size?" --graph

evaluate

Baseline vs RAG evaluation with an independent LLM judge. Compares an answer without context against a RAG answer on accuracy, relevance, completeness, specificity, and verdict.

axon evaluate "<question>" [FLAGS]

Requires AXON_ACP_ADAPTER_CMD.

suggest

Suggest new documentation URLs to crawl based on existing indexed content.

axon suggest [focus] [FLAGS]

[focus] is an optional topic hint. Outputs ranked URL candidates.

ingest

Ingest external sources into Qdrant. Source type is auto-detected from the target. Async by default.

axon ingest <TARGET> [FLAGS]
axon ingest <SUBCOMMAND> [ARGS]

Auto-detection rules (first match wins):

Input pattern	Detected as
`r/subreddit` or `reddit.com/*`	Reddit
`@handle`, `youtube.com/`, `youtu.be/`, bare 11-char video ID	YouTube
`github.com/owner/repo` or `owner/repo` slug	GitHub

Key flags:

Flag	Default	Description
`--wait <bool>`	`false`	Block until ingestion completes.
`--collection <name>`	`cortex`	Target Qdrant collection.

GitHub-specific flags:

Flag	Default	Description
`--no-source`	`false`	Skip source-code file indexing.
`--max-issues <n>`	`100`	Maximum issues to fetch (0 = unlimited).
`--max-prs <n>`	`100`	Maximum pull requests to fetch (0 = unlimited).

Reddit-specific flags:

Flag	Default	Description
`--sort <sort>`	`hot`	Post sort: `hot`, `top`, `new`, `rising`.
`--time <range>`	`day`	Time range for `top` sort: `hour`, `day`, `week`, `month`, `year`, `all`.
`--max-posts <n>`	`25`	Maximum posts to fetch.
`--min-score <n>`	`0`	Minimum score threshold for posts and comments.
`--depth <n>`	`2`	Comment traversal depth.
`--scrape-links`	off	Scrape content of linked URLs in link posts.

Job subcommands: status, cancel, errors, list, cleanup, clear, recover, worker.

axon ingest rust-lang/rust                                        # GitHub slug
axon ingest https://github.com/anthropics/claude-code --wait true
axon ingest tokio-rs/tokio --no-source --wait true               # skip source files
axon ingest "https://www.youtube.com/watch?v=..." --wait true
axon ingest @SpaceinvaderOne                                     # YouTube @handle
axon ingest r/unraid --sort top --time week
axon ingest "https://www.reddit.com/r/rust/" --wait true

sessions

Ingest local AI session history (Claude, Codex, Gemini) into Qdrant.

axon sessions [FLAGS]
axon sessions <SUBCOMMAND> [ARGS]

Provider paths:

Provider	Path
Claude	`~/.claude/projects/`
Codex	`~/.codex/sessions/`
Gemini	`~/.gemini/history/`, `~/.gemini/tmp/`

Flag	Default	Description
`--claude`	off	Include Claude sessions.
`--codex`	off	Include Codex sessions.
`--gemini`	off	Include Gemini sessions.
`--project <name>`	—	Case-insensitive substring filter on project name.
`--wait <bool>`	`false`	Block until ingestion completes.

If none of --claude, --codex, --gemini is set, all providers are ingested.

Job subcommands: status, cancel, errors, list, cleanup, clear, recover, worker.

axon sessions                                         # all providers, async
axon sessions --codex --wait true                     # Codex only, sync
axon sessions --claude --gemini --project axon        # filtered by project name

refresh

Revalidate already-indexed URLs. Checks ETag and content hash; re-embeds only changed content. Async by default.

axon refresh <url...> [FLAGS]
axon refresh <SUBCOMMAND> [ARGS]

One-off refresh:

axon refresh https://docs.rs/spider --wait true
axon refresh https://docs.rs/spider https://qdrant.tech/documentation
axon refresh https://docs.rs --wait true                 # manifest fallback

Job subcommands: status, cancel, errors, list, cleanup, clear, recover, worker.

Schedule subcommands:

axon refresh schedule add <name> [seed_url] [--every-seconds N | --tier high|medium|low] [--urls "u1,u2"]
axon refresh schedule add github:<owner>/<repo> [--tier high|medium|low]
axon refresh schedule list
axon refresh schedule enable <name>
axon refresh schedule disable <name>
axon refresh schedule delete <name>
axon refresh schedule worker
axon refresh schedule run-due [--batch <n>]       # default: 25

Tier presets:

Tier	Interval
`high`	1800 seconds (30 min)
`medium`	21600 seconds (6 hours)
`low`	86400 seconds (24 hours)

GitHub schedules check pushed_at before running — no re-ingest if nothing was pushed.

axon refresh schedule add docs-high https://docs.rs --tier high
axon refresh schedule add github:rust-lang/rust --tier medium
axon refresh schedule run-due --batch 25 --json

watch

Scheduled task management. Recurring definitions backed by axon_watch_defs and axon_watch_runs tables.

axon watch <SUBCOMMAND> [ARGS]

Implemented subcommands:

axon watch create <name> --task-type <type> --every-seconds <n> [--task-payload <json>]
axon watch list
axon watch run-now <id>
axon watch history <id> [--limit <n>]

Schema-defined but not yet implemented:

axon watch get <id>
axon watch update <id> [--every-seconds <n>]
axon watch pause <id>
axon watch resume <id>
axon watch delete <id>
axon watch artifacts <run_id>

axon watch with no subcommand defaults to list.

Task payload for task_type=refresh:

{"urls":["https://example.com/docs","https://example.com/api"]}

axon watch create docs-refresh \
  --task-type refresh \
  --every-seconds 300 \
  --task-payload '{"urls":["https://docs.rs/spider"]}'
axon watch list --json
axon watch run-now <uuid>
axon watch history <uuid> --limit 20

Note: AXON_LITE=1 disables the watch scheduler. Scheduler automation requires full mode.

graph

Knowledge graph operations backed by Neo4j. Requires AXON_NEO4J_URL.

axon graph build [<url>] [--url <url>] [--domain <domain>] [--all]
axon graph status
axon graph explore <entity>
axon graph stats
axon graph worker

Build requires at least one of: a URL, --domain, or --all.

axon graph worker runs the background extractor that pulls Qdrant chunks and writes entity/relation graphs to Neo4j.

axon graph build --url https://tokio.rs/tokio/tutorial
axon graph build --all
axon graph status
axon graph explore "Tokio"
axon graph stats

sources

List all indexed URLs with chunk counts.

axon sources [FLAGS]

domains

List indexed domains with document and chunk statistics.

axon domains [FLAGS]

stats

Show Qdrant collection statistics (point count, vector size, indexed segments).

axon stats [FLAGS]

status

Show async job queue status across all job families.

axon status [FLAGS]

doctor

Diagnose service connectivity. Checks Postgres, Redis, RabbitMQ, Qdrant, TEI, and LLM endpoint reachability.

axon doctor [FLAGS]

debug

Run doctor plus LLM-assisted troubleshooting with recommendations. Requires AXON_ACP_ADAPTER_CMD.

axon debug [FLAGS]

export

Export a full index manifest (jobs, ingest targets, refresh schedules, Qdrant summary) to JSON.

axon export [FLAGS]

Flag	Default	Description
`--include-history`	`false`	Include historical job records.

Not available in lite mode.

artifacts

Inspect MCP tool response artifacts stored under .cache/axon-mcp/. Used primarily via the MCP server but also accessible directly.

axon artifacts list
axon artifacts head <path> [--limit <n>]
axon artifacts grep <path> --pattern "<regex>" [--context-lines <n>]
axon artifacts wc <path>
axon artifacts read <path> [--pattern "<regex>"] [--full]
axon artifacts search --pattern "<regex>"
axon artifacts delete <path>
axon artifacts clean --max-age-hours <n> [--dry-run]

migrate

Migrate an unnamed-vector Qdrant collection to named-mode, enabling hybrid RRF search. One-time operation; no re-embedding required.

axon migrate --from <source> --to <destination>

Flag	Required	Description
`--from <name>`	Yes	Source collection (must use unnamed-vector schema).
`--to <name>`	Yes	Destination collection (auto-created if absent).

axon migrate --from cortex --to cortex_v2
# then update .env: AXON_COLLECTION=cortex_v2
# then restart all workers

Progress is logged every 100 pages (~25,600 points). At 2.57M points, expect 1–2 hours.

serve

Start the local app stack supervisor. Manages the WebSocket bridge backend, MCP HTTP server, all workers, shell server, and Next.js (port 49010).

axon serve [--port <n>]   # default port 49000

The web UI is served separately by apps/web at port 49010.

cargo run --bin axon -- serve
# or:
just serve

mcp

Start the MCP server. Default transport is HTTP (--transport http). See MCP Server for full details.

axon mcp [--transport stdio|http|both]

Job Lifecycle Subcommands

All async job families (crawl, extract, embed, ingest, sessions, refresh, graph) share these subcommands:

Subcommand	Description
`status <job_id>`	Show current state and metadata for a job.
`cancel <job_id>`	Cancel a pending or running job.
`errors <job_id>`	Show error text for a failed job.
`list`	List recent jobs (last 50).
`cleanup`	Remove failed, canceled, and old completed jobs.
`clear`	Delete all jobs and purge the queue. Requires `--yes` or prompts for confirmation.
`recover`	Reclaim stale or interrupted running jobs.
`worker`	Run a worker inline (blocking).

Job state machine: pending → running → completed | failed | canceled

MCP Server

axon mcp exposes a single MCP tool named axon with action/subaction routing.

Transport Modes

axon mcp                           # HTTP only (default)
axon mcp --transport stdio         # stdio only (for Claude Desktop)
axon mcp --transport http          # HTTP only
axon mcp --transport both          # stdio + HTTP concurrently

HTTP endpoint: http://<AXON_MCP_HTTP_HOST>:<AXON_MCP_HTTP_PORT>/mcp

Environment equivalents:

AXON_MCP_TRANSPORT=stdio           # or http / both
AXON_MCP_HTTP_HOST=0.0.0.0        # default
AXON_MCP_HTTP_PORT=8001            # default

Authentication is handled at the ingress layer (OAuth gateway + SWAG reverse proxy). The /mcp endpoint is unauthenticated at the application level.

Tool Contract

Tool name: axon Primary route field: action Lifecycle route: action + subaction Response field: response_mode (default: path)

Parser rules (strict):

action is required and must match canonical names exactly
subaction is required for lifecycle families
No fallback fields (command, op, operation)
No token normalization or case folding

Canonical success envelope:

{
  "ok": true,
  "action": "<resolved action>",
  "subaction": "<resolved subaction>",
  "data": { "...": "..." }
}

Direct Actions

These actions do not require subaction:

Action	Optional Fields
`ask`	`query`, `graph`, `diagnostics`, `collection`, `since`, `before`, `response_mode`
`elicit_demo`	`message`, `response_mode`
`export`	`include_history`, `response_mode`
`help`	`response_mode`
`map`	`url`, `limit`, `offset`, `response_mode`
`query`	`query`, `limit`, `offset`, `collection`, `since`, `before`, `response_mode`
`research`	`query`, `limit`, `offset`, `search_time_range`, `response_mode`
`retrieve`	`url`, `max_points`, `response_mode`
`scrape`	`url`, `render_mode`, `format`, `embed`, `response_mode`, `root_selector`, `exclude_selector`
`screenshot`	`url`, `full_page`, `viewport`, `output`, `response_mode`
`search`	`query`, `limit`, `offset`, `search_time_range`, `response_mode`

Lifecycle Families

These actions require subaction:

Family	Subactions
`artifacts`	`head\|grep\|wc\|read\|list\|delete\|clean\|search`
`crawl`	`start\|status\|cancel\|list\|cleanup\|clear\|recover`
`embed`	`start\|status\|cancel\|list\|cleanup\|clear\|recover`
`extract`	`start\|status\|cancel\|list\|cleanup\|clear\|recover`
`graph`	`build\|status\|explore\|stats`
`ingest`	`start\|status\|cancel\|list\|cleanup\|clear\|recover`
`refresh`	`start\|status\|cancel\|list\|cleanup\|clear\|recover\|schedule`

Refresh schedule subaction (schedule_subaction): list, create, delete, enable, disable

Also available as direct actions (no subaction): doctor, domains, sources, stats, status

Example requests:

{ "action": "scrape", "url": "https://example.com" }
{ "action": "crawl", "subaction": "start", "urls": ["https://example.com"] }
{ "action": "ingest", "subaction": "start", "source_type": "github", "target": "rust-lang/rust" }
{ "action": "ask", "query": "how does hybrid search work?" }
{ "action": "graph", "subaction": "build", "url": "https://tokio.rs" }
{ "action": "refresh", "subaction": "schedule", "schedule_subaction": "list" }
{ "action": "artifacts", "subaction": "list" }

Response Modes

All actions accept a response_mode field:

Mode	Behavior
`path` (default)	Write payload to artifact file. Response returns compact metadata: `path`, `bytes`, `line_count`, `sha256`, `preview`.
`inline`	Return payload directly in the response (truncated to safe size).
`both`	Write artifact AND include inline payload.
`auto-inline`	System-assigned only. Any payload ≤ `AXON_INLINE_BYTES_THRESHOLD` (default: 8,192 bytes) is returned inline automatically with `"response_mode": "auto-inline"`.

Per-action overrides:

ask and research: always write an artifact AND include key_fields.answer / key_fields.summary in path-mode responses. The answer is always immediately readable.
scrape and retrieve: always return path mode. These payloads can be megabytes. Use artifacts head or artifacts grep to access content.

Artifact Inspection

Artifact responses are stored under .cache/axon-mcp/. Inspect them with the artifacts action.

Preferred order (least to most expensive):

Read the shape field in path-mode responses — summarizes key/value types and status distributions.
artifacts head path="<rel_path>" — first 25 lines. Quick orientation.
artifacts grep pattern="..." context_lines=N — regex search with context.
artifacts search pattern="..." — cross-artifact regex search.
artifacts read pattern="..." — filtered line dump.
artifacts read full=true — full paginated dump (explicit opt-in required).

All artifact access uses the relative_path field — this works for both local stdio and remote HTTP clients.

Cleanup:

{ "action": "artifacts", "subaction": "clean", "max_age_hours": 24, "dry_run": true }
{ "action": "artifacts", "subaction": "clean", "max_age_hours": 24, "dry_run": false }

max_age_hours is required. dry_run defaults to true. Files in screenshots/ are never deleted.

Claude Desktop Integration

.mcp.json example for stdio transport:

{
  "mcpServers": {
    "axon": {
      "command": "axon",
      "args": ["mcp", "--transport", "stdio"],
      "env": {
        "AXON_PG_URL": "postgresql://axon:postgres@127.0.0.1:53432/axon",
        "AXON_REDIS_URL": "redis://127.0.0.1:53379",
        "AXON_AMQP_URL": "amqp://axon:axonrabbit@127.0.0.1:45535/%2f",
        "QDRANT_URL": "http://127.0.0.1:53333",
        "TEI_URL": "http://YOUR_TEI_HOST:52000"
      }
    }
  }
}

HTTP transport example:

{
  "mcpServers": {
    "axon-http": {
      "url": "https://axon.example.com/mcp"
    }
  }
}

MCP server config for ACP sessions is read from ${AXON_DATA_DIR}/axon/mcp.json (or ~/.config/axon/mcp.json as fallback). The Web UI MCP settings page writes to this same file.

Configuration

Copy .env.example to .env and fill in values. Two env files are used:

.env — app runtime variables + shared Docker Compose interpolation
services.env — infrastructure container credentials

Environment Variables

Core Infrastructure

Variable	Required	Default	Description
`AXON_PG_URL`	Yes	`postgresql://axon:postgres@127.0.0.1:53432/axon`	PostgreSQL connection URL
`AXON_REDIS_URL`	Yes	`redis://127.0.0.1:53379`	Redis connection URL
`AXON_AMQP_URL`	Yes	`amqp://axon:axonrabbit@127.0.0.1:45535/%2f`	RabbitMQ AMQP URL
`QDRANT_URL`	Yes	`http://127.0.0.1:53333`	Qdrant REST API base URL
`TEI_URL`	Yes	—	Text Embeddings Inference base URL
`AXON_DATA_DIR`	No	`./data`	Persistent data root for container volumes
`AXON_COLLECTION`	No	`cortex`	Default Qdrant collection name
`AXON_SQLITE_PATH`	No	`$AXON_DATA_DIR/axon/jobs.db`	SQLite path (lite mode only)
`AXON_LITE`	No	`0`	Set to `1` to enable lite mode (SQLite, no external queue)

LLM and ACP

Variable	Required	Default	Description
`AXON_ACP_ADAPTER_CMD`	Yes (for ask/evaluate/research)	—	ACP adapter binary (e.g. `codex`)
`AXON_ACP_ADAPTER_ARGS`	No	—	Additional arguments for the ACP adapter
`OPENAI_BASE_URL`	No	—	OpenAI-compatible API base URL
`OPENAI_API_KEY`	No	—	API key for the LLM endpoint
`OPENAI_MODEL`	No	—	Model name override passed to the ACP adapter
`AXON_ACP_AUTO_APPROVE`	No	`true`	Auto-approve ACP tool permission requests

Search

Variable	Required	Default	Description
`TAVILY_API_KEY`	Yes (for search/research)	—	Tavily search API key

Graph (Neo4j)

Variable	Required	Default	Description
`AXON_NEO4J_URL`	No	—	Neo4j HTTP base URL (e.g. `http://127.0.0.1:7474`). Graph features disabled when empty.
`AXON_NEO4J_USER`	No	`neo4j`	Neo4j username
`AXON_NEO4J_PASSWORD`	No	—	Neo4j password
`AXON_GRAPH_CONCURRENCY`	No	`4`	Graph worker concurrency
`AXON_GRAPH_LLM_URL`	No	—	LLM endpoint for entity extraction (e.g. `http://localhost:11434`)
`AXON_GRAPH_LLM_MODEL`	No	`qwen3.5:2b`	Model for graph entity extraction
`AXON_GRAPH_SIMILARITY_THRESHOLD`	No	`0.75`	Min similarity score for `SIMILAR_TO` edges
`AXON_GRAPH_SIMILARITY_LIMIT`	No	`20`	Max similarity edges per document
`AXON_GRAPH_CONTEXT_MAX_CHARS`	No	`2000`	Max characters for graph context injected into `ask --graph`
`AXON_GRAPH_TAXONOMY_PATH`	No	—	Path to custom taxonomy file

Ingest Credentials

Variable	Required	Default	Description
`GITHUB_TOKEN`	No	—	GitHub personal access token (raises rate limit from 60 to 5000 req/hr)
`REDDIT_CLIENT_ID`	Yes (Reddit)	—	Reddit OAuth2 app client ID
`REDDIT_CLIENT_SECRET`	Yes (Reddit)	—	Reddit OAuth2 app client secret

Note: yt-dlp must be on PATH for YouTube ingest targets.

Browser Rendering

Variable	Required	Default	Description
`AXON_CHROME_REMOTE_URL`	No	—	Chrome headless_browser management API URL (e.g. `http://axon-chrome:6000`)

Queue Names

Variable	Default	Description
`AXON_CRAWL_QUEUE`	`axon.crawl.jobs`	Crawl job queue name
`AXON_EXTRACT_QUEUE`	`axon.extract.jobs`	Extract job queue name
`AXON_EMBED_QUEUE`	`axon.embed.jobs`	Embed job queue name
`AXON_INGEST_QUEUE`	`axon.ingest.jobs`	Ingest job queue name
`AXON_REFRESH_QUEUE`	`axon.refresh.jobs`	Refresh job queue name
`AXON_GRAPH_QUEUE`	`axon.graph.jobs`	Graph job queue name

Worker Tuning

Variable	Default	Description
`AXON_INGEST_LANES`	`2`	Parallel ingest worker lanes
`AXON_EMBED_DOC_TIMEOUT_SECS`	`300`	Per-document embed timeout in seconds
`AXON_EMBED_STRICT_PREDELETE`	`true`	Delete existing points before re-embedding
`AXON_JOB_STALE_TIMEOUT_SECS`	`300`	Seconds before a running job is considered stale
`AXON_JOB_STALE_CONFIRM_SECS`	`60`	Grace period before stale job reclaim
`AXON_MAX_PENDING_CRAWL_JOBS`	`100`	Max pending crawl jobs before rejection (0 = unlimited)
`AXON_CRAWL_SIZE_WARN_THRESHOLD`	`10000`	Warn when uncapped crawl exceeds this page count (0 = disable)

Ask / RAG Tuning

Variable	Default	Description
`AXON_ASK_MIN_RELEVANCE_SCORE`	`0.45`	Minimum relevance score for candidate chunks
`AXON_ASK_CANDIDATE_LIMIT`	`64`	Candidate pool size from Qdrant
`AXON_ASK_CHUNK_LIMIT`	`10`	Chunks included in the final LLM context
`AXON_ASK_FULL_DOCS`	`4`	Number of top documents for additional chunk backfill
`AXON_ASK_MAX_CONTEXT_CHARS`	`120000`	Max LLM context characters
`AXON_ASK_MIN_CITATIONS_NONTRIVIAL`	`2`	Min unique citations for non-trivial answers
`AXON_ASK_AUTHORITATIVE_DOMAINS`	—	Comma-separated domains to boost in reranking
`AXON_ASK_AUTHORITATIVE_BOOST`	`0.0`	Score boost for authoritative-domain matches
`AXON_ASK_AUTHORITATIVE_ALLOWLIST`	—	Strict domain allowlist for retrieval candidates

MCP Transport

Variable	Default	Description
`AXON_MCP_TRANSPORT`	`http`	Transport mode: `stdio`, `http`, `both`
`AXON_MCP_HTTP_HOST`	`0.0.0.0`	HTTP bind host
`AXON_MCP_HTTP_PORT`	`8001`	HTTP bind port
`AXON_INLINE_BYTES_THRESHOLD`	`8192`	Payload size below which auto-inline is triggered (0 = disable)

Ports

Variable	Default	Description
`AXON_SERVE_PORT`	`49000`	WebSocket bridge server port
`AXON_SERVE_HOST`	`127.0.0.1`	WebSocket bridge bind host
`AXON_WEB_DEV_PORT`	`49010`	Next.js dashboard port
`SHELL_SERVER_PORT`	`49011`	Shell server port

Web UI Security

Variable	Required	Description
`AXON_WEB_API_TOKEN`	No (but recommended)	Primary token. Gates `/api/*` and `/ws` when set.
`AXON_WEB_BROWSER_API_TOKEN`	No	Optional second token for `/api/*` only (does not gate `/ws`).
`NEXT_PUBLIC_AXON_API_TOKEN`	Required when `AXON_WEB_API_TOKEN` is set	Browser-exposed token. Must equal `AXON_WEB_BROWSER_API_TOKEN` when that is set, otherwise `AXON_WEB_API_TOKEN`.
`AXON_WEB_ALLOWED_ORIGINS`	No	CORS allowed origins.
`AXON_WEB_ALLOW_INSECURE_DEV`	`false`	Loopback bypass for `/api/*`. Never set to `true` in production.

OAuth (MCP OAuth gateway)

Variable	Description
`GOOGLE_OAUTH_CLIENT_ID`	Google OAuth2 client ID
`GOOGLE_OAUTH_CLIENT_SECRET`	Google OAuth2 client secret
`GOOGLE_OAUTH_REDIRECT_URI`	OAuth redirect URI
`GOOGLE_OAUTH_SCOPES`	Requested OAuth scopes
`GOOGLE_OAUTH_DCR_TOKEN`	Dynamic Client Registration token
`GOOGLE_OAUTH_REDIS_URL`	Redis URL for OAuth token storage

TEI (Text Embeddings Inference)

Variable	Default	Description
`TEI_HTTP_PORT`	`52000`	TEI container port
`TEI_EMBEDDING_MODEL`	`Qwen/Qwen3-Embedding-0.6B`	Embedding model ID
`TEI_MAX_CONCURRENT_REQUESTS`	`256`	Max concurrent embedding requests
`TEI_MAX_BATCH_TOKENS`	`163840`	Max tokens per batch
`TEI_MAX_BATCH_REQUESTS`	`256`	Max requests per batch
`TEI_MAX_CLIENT_BATCH_SIZE`	`128`	Max batch size per client request
`TEI_POOLING`	`last-token`	Pooling strategy
`TEI_TOKENIZATION_WORKERS`	`8`	Tokenization thread count
`HF_TOKEN`	—	HuggingFace token for model download

Infrastructure

Docker Compose Stack

The stack is split into two compose files sharing the axon bridge network:

File	Contents
`docker-compose.services.yaml`	Infrastructure services (Postgres, Redis, RabbitMQ, Qdrant, TEI, Chrome)
`docker-compose.yaml`	App containers (workers, web UI)
`docker-compose.gpu.yaml`	GPU override — layer on top of services file on NVIDIA hosts

Infrastructure services:

Service	Image	Port	Purpose
`axon-postgres`	postgres:17-alpine	`53432`	Job persistence and state
`axon-redis`	redis:8.2-alpine	`53379`	Queue state, cancellation flags
`axon-rabbitmq`	rabbitmq:4.0-management	`45535`	AMQP job queue delivery
`axon-qdrant`	qdrant/qdrant:v1.13.1	`53333` (REST), `53334` (gRPC)	Vector store
`axon-tei`	ghcr.io/huggingface/text-embeddings-inference	`52000`	Embedding generation (NVIDIA GPU)
`axon-chrome`	built from `docker/chrome/Dockerfile`	`9222` (CDP), `6000` (management)	Headless browser for JavaScript rendering
`axon-ollama`	ollama/ollama:0.6.5	`11434`	Optional local LLM

All ports are bound to 127.0.0.1 (loopback only) by default.

App containers (for Docker deployment):

Service	Port	Purpose
`axon-workers`	`49000`, `8001`	Workers + `axon serve` + MCP HTTP
`axon-web`	`49010`	Next.js dashboard

Start infrastructure:

docker compose -f docker-compose.services.yaml up -d
# GPU hosts:
docker compose -f docker-compose.services.yaml -f docker-compose.gpu.yaml up -d
# or: just services-up

Start app containers (Docker deployment):

just up
# or: ./scripts/rebuild-fresh.sh

Start local development (infra + supervisor process):

just dev
# equivalent to: just services-up && axon serve

Stop everything:

just down-all

Lite Mode

Lite mode runs without Postgres, Redis, or RabbitMQ. Jobs are stored in SQLite and workers run in-process.

AXON_LITE=1 axon scrape https://example.com
# or:
axon --lite scrape https://example.com

Supported in lite mode: scrape, crawl (sync), map, embed, query, ask, extract, ingest, search, research, sources, stats, doctor, MCP server.

Not supported in lite mode: graph, refresh (including scheduling), watch scheduler, export.

Ingest Sources

GitHub Ingest

Ingests source code (tree-sitter AST-aware chunking), issues, pull requests, and wiki.

Supported tree-sitter languages: Rust, Python, JavaScript, TypeScript, Go, Bash.

Source code is indexed by default. Use --no-source to skip code and ingest only documentation and issues.

axon ingest rust-lang/rust
axon ingest https://github.com/tokio-rs/tokio --wait true
axon ingest tokio-rs/tokio --no-source       # docs/issues/PRs/wiki only
axon ingest tokio-rs/tokio --max-issues 0 --max-prs 0   # source only

Requires GITHUB_TOKEN for rates above 60 req/hr.

Reddit Ingest

Ingests subreddit posts, comments, and linked URLs. Requires Reddit OAuth2 credentials.

axon ingest r/unraid --sort top --time week
axon ingest "https://www.reddit.com/r/rust/"
axon ingest r/programming --max-posts 100 --min-score 10 --depth 3

YouTube Ingest

Ingests video transcripts via yt-dlp. Supports video URLs, @channel handles, playlists, and bare 11-character video IDs. yt-dlp must be on PATH.

axon ingest "https://www.youtube.com/watch?v=dQw4w9WgXcQ" --wait true
axon ingest @SpaceinvaderOne
axon ingest dQw4w9WgXcQ                    # bare video ID

AI Sessions Ingest

Ingests local conversation history from Claude, Codex, and Gemini into Qdrant.

axon sessions                              # all providers
axon sessions --claude --project axon      # filtered by project name

Uses an incremental state tracker (axon_session_ingest_state) so re-runs skip already-indexed sessions.

Vector Storage and RAG

Collection Modes

Axon supports two Qdrant collection modes:

Mode	Schema	Search
Unnamed (legacy)	`"vectors": {"size": N}`	Dense-only cosine
Named (current)	`"vectors": {"dense": ..., "sparse": {"bm42": ...}}`	RRF hybrid (dense + BM42 sparse)

New collections are created in named mode automatically. Legacy collections use axon migrate to upgrade.

Hybrid Search

Named-mode collections use Reciprocal Rank Fusion (RRF) via the Qdrant /query endpoint when hybrid search is active. Falls back to dense-only when the sparse query is empty or hybrid is disabled. No re-embedding is needed for migration — BM42 sparse vectors are computed locally from chunk_text payload fields.

RAG Pipeline

Text chunking: 2,000 characters with 200-character overlap. Each chunk is one Qdrant point.

Embedding: TEI batches with auto-retry on HTTP 413 (payload too large), 429, and 5xx. Up to 5 attempts with exponential backoff (1s, 2s, 4s, 8s plus jitter).

The ask retrieval pipeline is described under ask. Key tuning variables: AXON_ASK_CANDIDATE_LIMIT, AXON_ASK_MIN_RELEVANCE_SCORE, AXON_ASK_CHUNK_LIMIT, AXON_ASK_MAX_CONTEXT_CHARS.

Graph Operations

Graph features require AXON_NEO4J_URL. They are fully opt-in and gracefully disabled when the variable is absent.

Neo4j is accessed via HTTP transactional Cypher (/db/neo4j/tx/commit). Use an HTTP base URL, not a Bolt DSN.

Neo4j data model:

Nodes: Document {url, source_type, collection}, Chunk {point_id, url, collection, chunk_index}, Entity {name, entity_type, confidence}
Relationships: (Chunk)-[:BELONGS_TO]->(Document), (Entity)-[:MENTIONED_IN]->(Chunk), (Entity)-[:RELATES_TO]->(Entity), (Document)-[:SIMILAR_TO]->(Document)

Graph worker pipeline:

Pull Qdrant chunks by URL
Extract entities via taxonomy (built-in or AXON_GRAPH_TAXONOMY_PATH)
Optionally run LLM extraction for ambiguous entities
Write documents, chunks, entities, and relations to Neo4j
Compute SIMILAR_TO edges using Qdrant recommend scores

The embed worker auto-enqueues graph jobs for newly embedded URLs when AXON_NEO4J_URL is set.

ask --graph behavior: after vector retrieval, taxonomy-scan top chunks for entity names, query Neo4j neighborhood for those entities, and prepend a bounded Graph Context: block to the LLM context. Falls back to vector-only on any Neo4j failure.

Configuration:

AXON_NEO4J_URL=http://127.0.0.1:7474
AXON_NEO4J_USER=neo4j
AXON_NEO4J_PASSWORD=your-password
AXON_GRAPH_CONTEXT_MAX_CHARS=2000

Browser Rendering

Axon supports two fetch modes:

Mode	Description
`http`	Pure HTTP with `reqwest`. Fast; no JavaScript execution.
`chrome`	Chrome DevTools Protocol (CDP). Full JavaScript execution.
`auto-switch` (default)	HTTP first. If >60% of crawled pages are thin (<200 chars) or total coverage is too low, retries with Chrome.

Chrome requires a running Chrome instance. The axon-chrome container provides a headless_browser manager:

Port 6000: management API (GET / returns "healthy", /fork, /shutdown)
Port 9222: CDP proxy

Set AXON_CHROME_REMOTE_URL to point at the management API (e.g. http://axon-chrome:6000).

Use chrome mode explicitly for Single-Page Applications and sites that require JavaScript execution. Use http mode for static sites to minimize resource usage.

Performance Profiles

Set with --performance-profile <name> or the default high-stable.

Profile	Crawl concurrency	Sitemap concurrency	Backfill concurrency	Timeout	Retries	Backoff
`high-stable` (default)	CPUs×8 (64–192)	CPUs×12 (64–256)	CPUs×6 (32–128)	20s	2	250ms
`balanced`	CPUs×4 (32–96)	CPUs×6 (32–128)	CPUs×3 (16–64)	30s	2	300ms
`extreme`	CPUs×16 (128–384)	CPUs×20 (128–512)	CPUs×10 (64–256)	15s	1	100ms
`max`	CPUs×24 (256–1024)	CPUs×32 (256–1536)	CPUs×20 (128–1024)	12s	1	50ms

Override individual limits with --crawl-concurrency-limit, --sitemap-concurrency-limit, --backfill-concurrency-limit, --request-timeout-ms, --fetch-retries, and --retry-backoff-ms.

Web UI

The Next.js dashboard runs at http://localhost:49010 (port AXON_WEB_DEV_PORT).

Features:

Omnibox command interface (press / to focus)
Crawl, scrape, embed, query, and ask commands via WebSocket
Pulse workspace: AI-assisted research with conversation context
Results panel with job output streaming
MCP server settings page (/api/mcp)
Docker container stats in the status bar

The web UI connects to the axum WebSocket bridge at AXON_BACKEND_URL (default port 49000). The bridge provides:

/ws — command execution and Docker stats broadcast
/ws/shell — PTY bridge for terminal UI (loopback only)
/output/{*path} — serve generated output files
/download/{job_id}/pack.md|pack.xml|archive.zip — crawl artifact download

WebSocket authentication is controlled by AXON_WEB_API_TOKEN. The gate is disabled (open) when the variable is not set.

Security

URL Validation

All user-provided URLs are validated before any network request:

Scheme allowlist: http and https only
Blocked hosts: localhost, .localhost, .internal, .local
Blocked IP ranges: loopback, link-local, RFC-1918 private ranges, IPv4-mapped IPv6

DNS rebinding is mitigated via SsrfBlockingResolver, which re-checks resolved IPs at TCP connect time.

WebSocket Authentication

The /ws gate activates when AXON_WEB_API_TOKEN is set. Unset means open (trusted-network deployments). The browser sends the token as ?token= on the WebSocket URL. MCP OAuth clients (atk_ tokens) use /mcp only.

ACP Permission Security

Concurrent ACP sessions are isolated by (session_id, tool_call_id) composite keys. One session cannot receive another session's permission prompt. Auto-approve is on by default (AXON_ACP_AUTO_APPROVE=true). Set to false to require explicit UI approval.

Destructive Operations

The following operations are unauthenticated at the application level (network-level controls apply):

axon crawl clear, axon extract clear, axon refresh clear
axon crawl cancel <id>, axon refresh cancel <id>

Axon is designed for self-hosted single-user operation. All service ports are loopback-bound by default.

Never commit .env. Always set AXON_WEB_ALLOW_INSECURE_DEV=false in any network-accessible deployment.

Development

Build and Test

just setup          # bootstrap environment (once)
just check          # cargo check
just test           # run unit tests (excludes e2e)
just test-fast      # lib tests only
just fmt            # cargo fmt
just clippy         # cargo clippy
just build          # release build
just verify         # fmt-check + clippy + check + test (pre-PR gate)
just fix            # cargo fmt + clippy --fix

Local Development Stack

just services-up    # start infra (Postgres, Redis, RabbitMQ, Qdrant, TEI, Chrome)
just dev            # infra + axon serve supervisor (builds debug binary)
just serve          # axon serve only (infra must already be running)
just workers        # run all workers inline
just stop           # kill running serve, workers, and Next.js processes

Docker Deployment

just up             # build + start app containers
just down           # stop app containers
just down-all       # stop everything (app + infra)
just rebuild-fresh  # clean rebuild and restart

MCP Smoke Tests

just mcp-smoke
# Runs both full mode (AXON_LITE=0) and lite mode (AXON_LITE=1) suites

Manual mcporter commands:

mcporter --config config/mcporter.json call axon.axon action:help response_mode:inline --output json
mcporter --config config/mcporter.json call axon.axon action:doctor --output json
mcporter --config config/mcporter.json call axon.axon action:scrape url:https://www.rust-lang.org --output json
mcporter --config config/mcporter.json call axon.axon action:query query:'rust mcp sdk' --output json
mcporter --config config/mcporter.json call axon.axon action:crawl subaction:list limit:5 offset:0 --output json
mcporter --config config/mcporter.json call axon.axon action:artifacts subaction:list --output json

Web UI Development

just web-dev        # Next.js dev server
just web-build      # production build
just web-lint       # lint

Wrapper Script

Use ./scripts/axon instead of the cargo binary for local dev — it auto-sources .env:

./scripts/axon doctor
./scripts/axon scrape https://example.com --wait true
./scripts/axon mcp

End-to-End Workflow Example

Ingest a repository, embed it, and ask questions with graph context.

# 1. Start infrastructure
just services-up

# 2. Run the local supervisor (starts workers and web UI)
just dev

# 3. Verify connectivity
axon doctor

# 4. Ingest the tokio repository (source code + issues + PRs, async)
axon ingest tokio-rs/tokio

# 5. Check ingest job status
axon ingest list
axon ingest status <job_id>

# 6. Wait for ingest to complete (or use --wait true in step 4)
# Embed happens automatically after ingest

# 7. Verify content is indexed
axon sources
axon stats

# 8. Basic semantic search
axon query "async runtime architecture" --limit 5

# 9. RAG question answering
axon ask "how does tokio handle task scheduling?"

# 10. Build graph from indexed content (requires Neo4j)
axon graph build --all
axon graph worker   # run in background or separate terminal

# 11. Ask with graph context
axon ask "what are the key entities in the tokio runtime?" --graph

# 12. Schedule periodic refresh
axon refresh schedule add github:tokio-rs/tokio --tier medium

# 13. Set up a watch definition for recurring refresh
axon watch create tokio-refresh \
  --task-type refresh \
  --every-seconds 21600 \
  --task-payload '{"urls":["https://tokio.rs/tokio/tutorial"]}'

# 14. View indexed domains and sources
axon domains
axon sources

Related Files

File	Contents
`CLAUDE.md`	Canonical developer reference: commands, architecture, gotchas
`docs/ARCHITECTURE.md`	Subsystem map and data-flow diagrams
`docs/MCP.md`	MCP runtime and design guide
`docs/MCP-TOOL-SCHEMA.md`	MCP tool schema source of truth
`docs/GRAPH.md`	Graph features, Neo4j data model, and CLI reference
`docs/SECURITY.md`	Security model, SSRF controls, ACP permission isolation
`docs/JOB-LIFECYCLE.md`	Async job state machine, worker architecture
`docs/DEPLOYMENT.md`	Deploy and rollback procedures
`docs/SERVE.md`	WebSocket bridge architecture
`docs/TESTING.md`	Test strategy and mcporter smoke harness
`docs/commands/`	Per-command reference documentation
`docs/ingest/`	Per-source ingest deep dives
`CHANGELOG.md`	Release history

Related plugins

Plugin	Category	Description
homelab-core	core	Core agents, commands, skills, and setup/health workflows for homelab management.
overseerr-mcp	media	Search movies and TV shows, submit requests, and monitor failed requests via Overseerr.
unraid-mcp	infrastructure	Query, monitor, and manage Unraid servers: Docker, VMs, array, parity, and live telemetry.
unifi-mcp	infrastructure	Monitor and manage UniFi devices, clients, firewall rules, and network health.
gotify-mcp	utilities	Send and manage push notifications via a self-hosted Gotify server.
swag-mcp	infrastructure	Create, edit, and manage SWAG nginx reverse proxy configurations.
synapse-mcp	infrastructure	Docker management (Flux) and SSH remote operations (Scout) across homelab hosts.
arcane-mcp	infrastructure	Manage Docker environments, containers, images, volumes, networks, and GitOps via Arcane.
syslog-mcp	infrastructure	Receive, index, and search syslog streams from all homelab hosts via SQLite FTS5.
plugin-lab	dev-tools	Scaffold, review, align, and deploy homelab MCP plugins with agents and canonical templates.

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 1,274 Commits
.cargo		.cargo
.claude		.claude
.codex-plugin		.codex-plugin
.github/workflows		.github/workflows
apps/web		apps/web
config		config
crates		crates
docker		docker
docs		docs
migrations		migrations
scripts		scripts
tests		tests
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
.monolith-allowlist		.monolith-allowlist
ACP-GAP-ANALYSIS.md		ACP-GAP-ANALYSIS.md
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
GEMINI.md		GEMINI.md
Justfile		Justfile
README.md		README.md
axon.json		axon.json
axon.schema.json		axon.schema.json
crates.rs		crates.rs
deny.toml		deny.toml
docker-compose.services.yaml		docker-compose.services.yaml
docker-compose.test.yaml		docker-compose.test.yaml
docker-compose.yaml		docker-compose.yaml
lefthook.yml		lefthook.yml
lib.rs		lib.rs
main.rs		main.rs
renovate.json		renovate.json
rust-toolchain.toml		rust-toolchain.toml

Folders and files

Latest commit

History

Repository files navigation

Axon

Table of Contents

Overview

Installation

Prerequisites

Bootstrap

Build

Install to PATH

CLI Reference

Global Flags

Core Behavior

Crawl and Scrape

Output

Vector and Embedding

Performance Tuning

Service URL Overrides

Queue Configuration

scrape

crawl

map

extract

search

research

embed

query

retrieve

ask

evaluate

suggest

ingest

sessions

refresh

watch

graph

sources

domains

stats

status

doctor

debug

export

artifacts

migrate

serve

mcp

Job Lifecycle Subcommands

MCP Server

Transport Modes

Tool Contract

Direct Actions

Lifecycle Families

Response Modes

Artifact Inspection

Claude Desktop Integration

Configuration

Environment Variables

Core Infrastructure

LLM and ACP

Search

Graph (Neo4j)

Ingest Credentials

Browser Rendering

Queue Names

Worker Tuning

Ask / RAG Tuning

MCP Transport

Ports

Web UI Security

OAuth (MCP OAuth gateway)

TEI (Text Embeddings Inference)

Infrastructure

Docker Compose Stack

Lite Mode

Ingest Sources

GitHub Ingest

Reddit Ingest

Packages