refactor: split web into separate web/api projects with shared backend package

[e7h4n]

Background

Currently, turbo/apps/web contains both frontend UI and backend API code in a single package. This means any change to the web app - including UI-only changes like landing page copy or styling - triggers the full cli-e2e test suite (~20 minutes).

Root cause: The CI uses turbo-ignore for change detection, which treats the entire web package as one unit.

Impact: Content/style PRs that don't affect CLI or runner functionality still wait 20+ minutes for cli-e2e to complete.

Proposed Solution: Full Package Extraction (Approach A)

Split into three packages:

turbo/
├── apps/
│   ├── api/           # NEW: API routes only (/api/*, /v1/*)
│   └── web/           # Frontend pages only
└── packages/
    └── backend/       # NEW: Shared backend code
        ├── src/
        │   ├── db/    # Schema (15 tables) + migrations (49 files)
        │   ├── lib/   # Business logic services (~68 files)
        │   ├── env.ts
        │   └── types/
        └── package.json

Expected CI Behavior After Split

Change Type	cli-e2e Triggered?
web/ (UI pages, components, i18n)	No
api/ (API routes)	Yes
packages/backend/	Yes
packages/core/	Yes

Implementation Phases

Phase 1: Create @vm0/backend Package

• Extract /src/db/ (schema, migrations)
• Extract /src/lib/ (all services)
• Extract /src/env.ts and /src/types/
• Update web to import from @vm0/backend
• Verify existing tests pass

Phase 2: Create turbo/apps/api

• Create new Next.js app for API
• Move /app/api/* routes (34 files)
• Move /app/v1/* routes (21 files)
• Configure to use @vm0/backend
• Set up Vercel project for api.vm0.dev

Phase 3: Clean Up Web App

• Remove API-only dependencies from web
• Update CI change detection for api-changed
• Update cli-e2e to depend on deploy-api instead of deploy-web
• Clean up unused code

Technical Considerations

Files to Move

To packages/backend/:

• src/db/schema/*.ts (15 tables)
• src/db/migrations/ (49 files)
• src/lib/**/*.ts (~68 files including services)
• src/env.ts
• src/types/

To apps/api/:

• app/api/**/*.ts (34 routes)
• app/v1/**/*.ts (21 routes)

Stays in apps/web/:

• app/[locale]/* pages
• app/sign-in/, app/sign-up/
• app/cli-auth/
• app/components/
• messages/ (i18n)
• public/ (assets)

API URL Strategy

Use subdomain approach: api.vm0.dev

• Requires new Vercel project (VERCEL_PROJECT_ID_API)
• CLI and runner need to use new API base URL
• May need CORS configuration

Special Cases

1. CLI Auth Flow: /cli-auth/page.tsx is a frontend page that calls API endpoints. The page stays in web, API endpoints move to api.

2. Database Migrations: Should be owned by @vm0/backend package with a separate db:migrate script.

3. Global Services Pattern: initServices() must be exported from @vm0/backend and work in both apps.

Research Documents

Full analysis available at:

• /tmp/deep-dive/web-api-split/research.md
• /tmp/deep-dive/web-api-split/innovate.md

---

Created from conversation context using /issue-create

[e7h4n]

🔬 Research Phase

Executive Summary

The turbo/apps/web application is a Next.js monolith containing both frontend pages and backend API routes. This research documents the current architecture and technical challenges.

Current Architecture

Component	Count	Description
Frontend Pages	11	Next.js App Router pages
Internal API Routes	34	/app/api/* endpoints
Public API Routes	17	/app/v1/* endpoints
Backend Library Files	71	src/lib/ services
Database Tables	15	Drizzle ORM schemas
Migrations	49	Database migrations

API Routes Breakdown

Internal APIs (/app/api/):

• Agent management (11 routes) - runs, composes, sessions, checkpoints
• Storage (5 routes) - prepare, commit, download, list
• Webhooks (7 routes) - events, heartbeat, telemetry
• Infrastructure (11 routes) - auth, cli, scope, runners, cron

Public APIs (/app/v1/):

• Agents (3 endpoints)
• Runs (6 endpoints)
• Artifacts (4 endpoints)
• Volumes (4 endpoints)

Critical Shared Services

1. Global Services Singleton (init-services.ts) - Database connection management
2. Database Schema - 15 tables in src/db/schema/
3. Authentication - Clerk + sandbox tokens + runner auth
4. API Contracts - ts-rest definitions in @vm0/core

CI/CD Integration Points

• Change detection uses turbo-ignore per app
• deploy-web outputs preview-url for platform and cli-e2e
• Platform receives VITE_API_URL at build time

Technical Challenges

Challenge	Complexity
Database migration handling	High
CI/CD restructuring	High
Global services adaptation	High
Schema management	Medium
Authentication refactoring	Medium
Test infrastructure	Medium

Key Finding

The web app has clear separation between API routes and frontend pages at the file system level. The main challenges are shared backend code (71 files) and CI/CD orchestration.

---

Phase 1/3 of deep-dive workflow

[e7h4n]

💡 Innovation Phase

Proposed Approaches

Three approaches were evaluated for splitting web into separate web/api projects:

Approach A: Full Extraction with Shared Backend Package

turbo/
├── apps/
│   ├── api/           # API routes only
│   └── web/           # Frontend pages only
└── packages/
    └── backend/       # Shared backend code

Pros: Clean architecture, no duplication, clear ownership
Cons: High initial effort, complex testing setup

Approach B: Simple Duplication (Recommended)

Duplicate web → api, then gradually clean up each app in phases:

1. Duplicate web to api, update turbo/lefthook config
2. User creates Vercel project, update CI/CD
3. Configure release-please for api
4. User points api.vm0.ai domain
5. Update web to call api.vm0.ai
6. Remove API from web
7. Remove pages from api
8. Clean up unused code

Pros: Low risk, easy rollback, testable at each step
Cons: Temporary code duplication

Approach C: Hybrid

Start with duplication, extract @vm0/backend after migration complete.

Trade-off Analysis

Factor	A (Full)	B (Simple)	C (Hybrid)
Initial Complexity	High	Low	Low
Risk Level	Medium-High	Low	Low
Code Duplication	None	Temporary	Temporary
Time to First PR	Longer	Shorter	Shorter
Rollback Difficulty	Hard	Easy	Easy

Recommendation

Approach B (Simple Duplication) is recommended:

• Aligns with user's phased plan
• Low risk with easy rollback
• Can evolve to Approach A/C later

Technical Considerations

1. Database Migrations - api takes ownership in Approach B
2. API URL - Web needs NEXT_PUBLIC_API_URL for client components
3. CORS - api.vm0.ai must accept requests from www.vm0.ai
4. CLI E2E - Should depend on deploy-api after migration
5. Auth - Both apps share Clerk keys (no changes needed)

---

Phase 2/3 of deep-dive workflow

[e7h4n]

📋 Plan Phase

Phase 1: Duplicate Web to API

Following Approach B (Simple Duplication) - the user's 9-step phased plan.

This plan covers Step 1: Duplicate web to api project, modify turbo/lefthook configuration, create a PR that passes CI.

Task Breakdown

Task	Description	Files Affected
1	Copy web/ to api/	Create turbo/apps/api/
2	Update package.json	turbo/apps/api/package.json
3	Verify turbo config	No changes needed (glob auto-includes)
4	Verify lefthook config	No changes needed (glob auto-includes)
5	Update release-please	release-please-config.json, .release-please-manifest.json
6	Create CHANGELOG	turbo/apps/api/CHANGELOG.md
7	Update CI change detection	.github/workflows/turbo.yml
8	Verify test job	No changes needed
9	Final verification	Run lint, build, test

Dependency Graph

Task 1 (Copy directory)
    ├── Task 2 (Update package.json)
    │       ├── Task 3-4 (Verify configs)
    │       ├── Task 5 (Update release-please)
    │       └── Task 7 (Update CI)
    └── Task 6 (Create CHANGELOG)
            └── Task 8-9 (Verify & test)

Definition of Done

• turbo/apps/api/ directory exists
• api package.json has name "api"
• pnpm install succeeds
• pnpm turbo run build --filter=api succeeds
• pnpm turbo run lint passes
• pnpm check-types passes
• pnpm test passes
• release-please-config.json includes api
• .release-please-manifest.json includes api version
• CI workflow has api-changed detection
• PR created and CI passes

Risk Assessment

Risk	Likelihood	Mitigation
pnpm workspace issues	Low	Glob pattern auto-detects
CI timeout (doubled build)	Medium	Turbo caching helps
Test failures	Low	Tests are identical

Out of Scope (Phase 1)

• ❌ Vercel project creation (Phase 2)
• ❌ CI deployment jobs for api (Phase 2)
• ❌ Domain configuration (Phase 4)
• ❌ API URL redirection (Phase 5)
• ❌ Code removal (Phases 7-9)

Next Steps After Phase 1

1. User Action: Create Vercel project for api, provide project ID
2. Phase 2: Update CI/CD with deploy-api job

---

Phase 3/3 - Ready for approval

To proceed with implementation, approve this plan and run /issue-continue