Skip to content

README.md

Latest commit

 

History

History
119 lines (91 loc) · 3.45 KB

README.md

File metadata and controls

119 lines (91 loc) · 3.45 KB

auto-llm: Autonomous Multi-Agent Web Research System

auto-llm is a multi-agent, MCP-based research system that can run long-lived web exploration and synthesis workflows powered by OpenRouter models (Llama 3.3 70B), Playwright MCP, a filesystem MCP, and a zero-vector MCP for semantic memory.

It is designed to:

  • Explore the web via Playwright MCP
  • Discover interesting content using a scoring heuristic
  • Grow its own exploration frontier over time
  • Remember findings semantically via a vector DB
  • Synthesize patterns and insights into reports
  • Survive interruptions with checkpointed state and resume support

Features

  • Multi-agent architecture
    • scout: curiosity-driven web explorer
    • investigator: deep-dive analyst
    • synthesizer: pattern recognizer and report writer
  • MCP-native integration
    • Playwright MCP for browser automation
    • Filesystem MCP for durable storage
    • Zero-vector MCP for semantic memory
  • Checkpointing & resilience
    • Persistent checkpoint.json with visited URLs, cycle index, and stats
    • Safe to run overnight; resumes from where it left off
  • Dynamic frontier of URLs
    • Scout proposes new candidate URLs via structured FRONTIER_LINK lines
    • Frontier entries are scored, decayed over time, and domain-balanced
  • Structured logging
    • Console + file logging (data/state/research_agent.log)

Project Layout

  • main.py – CLI entrypoint & high-level orchestrator
  • agents/
    • scout.py – Curiosity Scout agent definition
    • investigator.py – Deep Investigator agent definition
    • synthesizer.py – Pattern Synthesizer agent definition
  • workflows/
    • exploration.py – Exploration cycles, checkpointing, dynamic frontier
    • synthesis.py – Synthesis passes and report generation
  • mcp_agent.config.yaml – MCP server configuration
  • mcp_agent.secrets.yaml – OpenRouter credentials (gitignored)
  • data/
    • observations/ – Saved observation JSON files
    • reports/ – Markdown reports (investigation + synthesis)
    • screenshots/ – Playwright screenshots
    • state/ – Checkpoint and logs
  • docs/
    • user-guide.md – Detailed user & developer documentation

Prerequisites

  • Python 3.10+
  • Node.js + npm (for npx and MCP servers)
  • OpenRouter API key

Quickstart

  1. Create and activate a virtualenv
cd auto-llm
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
  1. Configure OpenRouter

Edit mcp_agent.secrets.yaml:

openai:
  api_key: "sk-or-v1-YOUR_OPENROUTER_API_KEY"
  base_url: "https://openrouter.ai/api/v1"
  organization: null
  1. Smoke test: single exploration
python main.py explore --urls https://example.com
  1. Short multi-cycle run with synthesis
python main.py overnight \
  --urls https://news.ycombinator.com/newest https://github.com/trending \
  --cycles 3 \
  --synthesis-every 2
  1. One-off synthesis over recent data
python main.py synthesize --lookback-hours 24

Where to Go Next

  • Read docs/user-guide.md for a detailed explanation of:
    • Agent roles and behaviors
    • Frontier mechanics and scoring
    • Checkpoint schema and resume semantics
    • How to add new agents or MCP servers

This README is intentionally concise; the user guide contains the full system narrative and is the best starting point for new developers who want to understand or extend the system.