feat: add RemoteLLM for GPU-accelerated embedding

[NathanSkene]

Summary

• Adds a RemoteLLM class that calls a remote llama-server via HTTP for GPU-accelerated embedding
• Enabled by setting QMD_REMOTE_URL=http://host:port environment variable
• Reduces embedding time from hours to minutes for large document collections (~12k files in ~20 min on an A10 GPU)

How it works

When QMD_REMOTE_URL is set, getDefaultLlamaCpp() returns a RemoteLLM instance instead of a local LlamaCpp. Only embed/embedBatch calls go to the remote server — tokenization for chunking uses a local ~4 chars/token approximation to avoid thousands of serial HTTP round-trips.

Usage

# Start llama-server on GPU machine (Lambda Labs, Vast.ai, etc.)
llama-server -m embeddinggemma-300M-Q8_0.gguf --embedding --port 8080 -c 8192 -b 2048 -ub 2048

# SSH tunnel from local machine
ssh -L 8080:localhost:8080 ubuntu@gpu-host -N &

# Run QMD embed with remote backend
QMD_REMOTE_URL=http://localhost:8080 qmd embed

Design decisions

Decision	Rationale
Local tokenization (~4 chars/token)	Remote tokenize/detokenize made thousands of serial HTTP calls during chunking — reduced from ~40 min to near-instant
fetchWithRetry with exponential backoff	Long-running jobs over SSH tunnels need resilience to transient network failures
Stubbed generate/rerank/expandQuery	Only embedding is needed for qmd embed workflow
Single env var activation	Zero config changes needed — just set QMD_REMOTE_URL

Testing

Tested on 12,599 files / 21,970 chunks with 97.5% success rate. The ~2.5% failures are oversized chunks exceeding the server's 8192-token context window — expected and tolerable.

Changes

• src/llm.ts: +153 lines (RemoteLLM class + factory function update)

🤖 Generated with Claude Code

[brettdavies]

Hey @NathanSkene -- nice work on the remote embedding support! I just opened #153 which adds a file size limit to qmd embed (skips files over 5MB before tokenization). It touches src/qmd.ts only, so there should be zero conflict with your src/llm.ts changes.

One thing that might be relevant: your PR mentions ~2.5% of chunks fail due to exceeding the 8192-token context window. The size limit in #153 would reduce that by filtering out the largest files upfront. Could be complementary if both land.

I'm also planning a small follow-up PR to add gpuLayers: "max" to loadModel() calls for local GPU acceleration. Wanted to flag it early since it also touches src/llm.ts -- happy to rebase on top of yours if it merges first.