diff --git a/plan.md b/plan.md index edb9d136..9723c996 100644 --- a/plan.md +++ b/plan.md @@ -1,392 +1,313 @@ -# CLI Implementation Plan +# OAuth Connectors CLI Implementation Plan + +Reference issue: https://github.com/base44/cli/issues/184 ## Overview -Generate a comprehensive CLI tool that provides a unified interface for managing Base44 applications, entities, functions, deployments, and related services. - -## Core Architecture - -### Project Structure -- **Package**: `base44` - Single package published to npm -- **Core Module**: `src/core/` - Shared utilities, API clients, schemas, and configuration management -- **CLI Module**: `src/cli/` - CLI commands and entry point -- **Build System**: TypeScript compiler (`tsc`) for production builds -- **Package Manager**: npm for dependency management - -### CLI Framework -- **Technology**: TypeScript with Commander.js for CLI framework -- **Structure**: Command-based architecture with subcommands -- **CLI Name**: `base44` -- **User Prompts**: Use `@clack/prompts` for interactive user prompts -- **Package Distribution**: Support for multiple package managers - - Homebrew (brew) - macOS/Linux - - Scoop - Windows - - npm - Node.js ecosystem (package published to npm as `base44`) - -## Feature Implementation Plan - -### 1. Authentication & Identity -- **`base44 login`** - - Device-based authentication - - OAuth flow implementation - - Token storage and management - - Session persistence - -- **`base44 whoami`** - - Display current authenticated user - - Show account information - - Display active session details - -- **`base44 logout`** - - Clear authentication tokens - - Remove session data - - Logout from current device - -### 2. Entities Management -- **`base44 entities`** - - List all entities in the project - - Display entity hierarchy (e.g., entities → auth.group) - -- **`base44 entities pull`** - - Read schemas from remote - - Sync entity definitions - - Download entity configurations - - **Validate downloaded schemas using Zod** - Ensure schema integrity - -- **`base44 entities push`** - - Write schemas to remote - - Upload entity definitions - - **Validate before pushing using Zod schemas** - Local validation before upload - - Schema structure verification - -### 3. Functions Management -- **`base44 functions`** - - List all functions - - Display function hierarchy (e.g., functions → hello.IT) - - Show function metadata - -- **`base44 functions [function-name]`** - - View specific function details - - Show function code, triggers, and configuration - -- **Cron Jobs Support** - - Schedule management - - **Zod validation for cron expressions** - Validate cron syntax - - Job listing and status - -### 4. Development Environment -- **`base44 dev`** - - Start local development server - - Hot reload for functions - - Local testing environment - -- **`base44 dev --functions`** - - Development mode with function support - - Function hot-reloading - -### 5. Deployment -- **`base44 deploy`** - - Deploy entire application - - Environment selection (staging/production) - -- **`base44 deploy --client`** - - Deploy client-side application only - - Frontend build and deployment - -- **`base44 deploy --fullstack`** - - Deploy full-stack application - - Backend + Frontend deployment - - Database migrations - -### 6. Project Initialization -- **`base44 new`** / **`base44 init`** / **`base44 create`** - - Initialize new Base44 project - - Project scaffolding - -- **`base44 new --blank`** - - Create blank project template - - Minimal project structure - -- **`base44 new --example`** - - Create project from example template - - Pre-configured starter project - -### 7. Linking & Integration -- **`base44 link`** - - Link local project to remote Base44 project - - Establish connection between local and cloud - - Configure project association - -### 8. AI Features -- **`base44 ai`** - - AI-powered assistance - - Code generation and suggestions - -- **`base44 ai does [prompt]`** - - Execute AI commands - - Natural language to CLI actions - - Intelligent task automation - -### 9. Secrets Management -- **`base44 secrets`** - - List all secrets - - Display secret metadata (not values) - -- **`base44 secrets get [key]`** - - Retrieve specific secret value - - Secure secret retrieval - - Environment variable export option - -- **`base44 secrets set [key] [value]`** - - Set or update secret value - - Secure secret storage - - **Zod validation for secret keys and values** - Ensure proper format - - Encryption - - Interactive prompts for secret input using `@clack/prompts` - -### 10. Domains Management -- **`base44 domains`** - - List configured domains - - Show domain status and configuration - - Display SSL certificate information - -- **`base44 domains add [domain]`** - - Add new domain - - **Zod validation for domain format** - Ensure valid domain structure - - DNS configuration assistance - - Interactive domain setup using `@clack/prompts` - -- **`base44 domains remove [domain]`** - - Remove domain configuration - -## Implementation Phases - -### Phase 0: Skeleton -**Goal**: Set up the basic project structure and create placeholder commands for authentication. - -1. **Project Structure Setup** - - Create project folder structure: + +Add OAuth connectors as a CLI resource, allowing users to define connector configurations in `connectors/*.jsonc` files and push them to Base44 apps. + +--- + +## Task 1: Resource File Parsing + +### Objective +Read and validate connector JSONC files from the `connectors/` directory. + +### Subtasks + +1.1. **Define Zod schemas** + - `IntegrationTypeSchema` - enum of supported integration types: ``` - cli/ - ├── src/ - │ ├── core/ # Core module - │ │ ├── api/ # API client code - │ │ ├── config/ # Configuration management - │ │ ├── schemas/ # Zod schemas - │ │ ├── utils/ # Utility functions - │ │ ├── types/ # TypeScript type definitions - │ │ └── index.ts # Core module exports - │ └── cli/ # CLI module (main CLI) - │ ├── commands/ - │ │ └── auth/ - │ │ ├── login.ts - │ │ ├── whoami.ts - │ │ └── logout.ts - │ ├── utils/ # CLI-specific utilities - │ │ ├── index.ts - │ │ ├── packageVersion.ts - │ │ └── runCommand.ts - │ └── index.ts # Main CLI entry point (with shebang) - ├── dist/ # Build output - ├── package.json # Package configuration - ├── tsconfig.json # TypeScript configuration - ├── .gitignore - └── README.md + googlecalendar, googledrive, gmail, googlesheets, googledocs, googleslides, + slack, notion, salesforce, hubspot, linkedin, tiktok ``` - -2. **Build Process & Configuration** - - Set up TypeScript configuration (`tsconfig.json`) - - Configure output directory structure (`dist/cli/`) - - **ES Modules**: Package uses `"type": "module"` for ES module support - - **Development**: Use `tsx` for development/watch mode - - **Production**: Use `tsdown` to bundle all code and dependencies into single file - - **Zero Dependencies**: All packages bundled - -3. **Package.json Setup** - - **Package** (`base44`): - - All dependencies in `devDependencies` (bundled at build time): - - `zod` - Schema validation - - `commander` - CLI framework - - `@clack/prompts` - User prompts and UI components - - `chalk` - Terminal colors (Base44 brand color: #E86B3C) - - `json5` - JSONC/JSON5 config parsing - - `ky` - HTTP client - - `ejs` - Template rendering - - `globby` - File globbing - - `dotenv` - Environment variables - - Zero runtime `dependencies` - everything bundled - - Set up bin entry point for CLI executable (`./dist/cli/index.js`) - - Set up build and dev scripts - - **Shebang**: Main entry point (`src/cli/index.ts`) includes `#!/usr/bin/env node` - -4. **Authentication Commands (Implemented)** - - Create `base44 login` command - - Use Commander.js to register command - - Use `@clack/prompts` tasks for async operations - - Store auth data using `writeAuth` from `src/core/config/auth.js` - - Wrap with `runCommand` utility for consistent branding - - Create `base44 whoami` command - - Use Commander.js to register command - - Read auth data using `readAuth` from `src/core/config/auth.js` - - Display user information using `@clack/prompts` log - - Wrap with `runCommand` utility for consistent branding - - Create `base44 logout` command - - Use Commander.js to register command - - Delete auth data using `deleteAuth` from `src/core/config/auth.js` - - Wrap with `runCommand` utility for consistent branding - - Ensure all commands are properly registered in main CLI entry point - - Test that commands are accessible and show help text - -5. **Import Structure** - - Set up proper ES module imports/exports (`.js` extensions in imports) - - Create barrel exports for command modules if needed - - Ensure TypeScript path resolution works correctly - - Use ES module syntax throughout (`import`/`export`) - -**Deliverables**: -- ✅ Complete folder structure -- ✅ Working build process (tsc for production, tsx for development) -- ✅ Package.json with all scripts -- ✅ Three auth commands (login, whoami, logout) fully implemented -- ✅ CLI can be run and commands respond with help text -- ✅ Base44 branding via `runCommand` utility wrapper - -### Phase 1: Foundation -1. ✅ Implement Commander.js command framework -2. ✅ Integrate `@clack/prompts` for user interactions -3. ✅ Set up Zod schema validation infrastructure - - ✅ Create base schemas for auth data (`AuthDataSchema`) - - ✅ Create config file schemas - - ✅ Set up validation utilities -4. ✅ Create authentication system (`base44 login`, `base44 whoami`, `base44 logout`) - - ✅ Auth data stored in `~/.base44/auth/auth.json` - - ✅ Zod validation for auth data - - ✅ Cross-platform file system utilities - - ✅ Error handling with user-friendly messages -5. Package manager distribution setup (npm, brew, scoop) - -### Phase 2: Core Features -1. Entities management (`base44 entities`, `pull`, `push`) -2. Functions listing and management -3. Project initialization (`base44 new`, `base44 init`, `base44 create`) -4. Linking functionality (`base44 link`) - -### Phase 3: Development & Deployment -1. Development server (`base44 dev`) -2. Deployment commands (`base44 deploy --client`, `base44 deploy --fullstack`) -3. Cron job management integration - -### Phase 4: Advanced Features -1. Secrets management (`base44 secrets get`, `base44 secrets set`) -2. Domains management (`base44 domains`) -3. AI integration (`base44 ai`, `base44 ai does`) - -### Phase 5: Polish & Distribution -1. Error handling and validation -2. Help documentation and examples -3. Package distribution (brew, scoop, npm via GitHub Actions) -4. Testing and quality assurance - -## Technical Considerations - -### Configuration -- **Global Auth Config**: Stored in `~/.base44/auth/auth.json` (managed by `src/core/config/auth.ts`) -- Local config file (`.base44/config.json` or similar) - for project-specific settings -- Global config for user preferences -- Environment-specific settings -- **Zod schema validation for all configuration files** - Validate config structure and values -- Type-safe config access using Zod-inferred types -- **File System Utilities**: Cross-platform file operations in `src/core/utils/fs.ts` - -### API Integration -- REST API client for Base44 services (using `fetch` or `axios`) -- Authentication token management -- Rate limiting and retry logic -- Error handling and user feedback -- **Zod schema validation for all API responses** - Validate and type-check API responses at runtime -- TypeScript types generated from Zod schemas for type safety - -### Build & Distribution -- **Project Structure** - Single package with `core` and `cli` modules -- **ES Modules** - Package uses `"type": "module"` for native ES module support -- **Zero-Dependency Distribution** - All runtime dependencies bundled into single JS file -- **Build Tools**: - - Production: `tsdown` bundles all code and dependencies into `dist/cli/index.js` - - Development: `tsx` for fast watch mode and direct TypeScript execution - - Type checking: `tsc --noEmit` for validation without emitting files -- **CLI Entry Point**: `src/cli/index.ts` includes shebang (`#!/usr/bin/env node`) -- GitHub Actions for automated builds and npm releases - -### Security -- Secure credential storage -- Encrypted secret management -- Token refresh mechanisms -- **Zod-based input validation and sanitization** - Validate all user inputs and CLI arguments -- Schema validation for secrets and sensitive data - -### User Experience -- **Base44 Branding**: All commands wrapped with `runCommand` utility showing Base44 intro banner (color: #E86B3C) -- Clear error messages with try-catch error handling -- Progress indicators for long operations using `@clack/prompts` tasks -- Interactive prompts using `@clack/prompts` for better UX -- Comprehensive help system via Commander.js -- Spinners and loading states for async operations -- **Command Wrapper Pattern**: All commands use `runCommand()` utility for consistent branding and error handling - -## Schema Validation with Zod - -### API Response Validation -- Define Zod schemas for all API endpoints -- Validate API responses before processing -- Type-safe API client with inferred types from Zod schemas -- Clear error messages when API responses don't match expected schema -- Examples: - - `UserSchema` for authentication responses - - `EntitySchema` for entity definitions - - `FunctionSchema` for function metadata - - `DeploymentSchema` for deployment status - - `SecretSchema` for secrets management - - `DomainSchema` for domain configurations - -### Configuration File Validation -- Zod schemas for all configuration files: - - `.base44/config.json` - Project configuration - - Global config files - - Entity schema files - - Function configuration files -- Validate on read to catch configuration errors early -- Type-safe config access throughout the application - -### File Schema Validation -- Validate entity schema files before push operations -- Validate function definitions and configurations -- Validate project structure files -- Ensure data integrity before syncing with remote - -### Input Validation -- Validate CLI command arguments using Zod -- Validate user input from prompts -- Validate environment variables -- Validate secrets before storage + - `ConnectorResourceSchema` - object with `type` and `scopes` array + +1.2. **File structure** + ``` + connectors/ + ├── googlecalendar.jsonc + ├── slack.jsonc + └── notion.jsonc + ``` + +1.3. **Resource schema** + ```jsonc + // connectors/googlecalendar.jsonc + { + "type": "googlecalendar", + "scopes": [ + "https://www.googleapis.com/auth/calendar.readonly", + "https://www.googleapis.com/auth/calendar.events" + ] + } + ``` + +1.4. **Validation behavior** + | Scenario | Behavior | + |----------|----------| + | Unknown integration type | Error: reject file | + | Unknown scope for integration | Warning only (OAuth provider validates) | + | Empty scopes array | Warning (except Notion) | + | Missing `type` field | Error: reject file | + +1.5. **Implementation location** + - Create `src/resources/connectors/` directory + - Add `schema.ts` for Zod schemas + - Add `reader.ts` for file reading logic + +--- + +## Task 2: API Client + +### Objective +Add methods to communicate with apper backend endpoints. + +### Endpoints + +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/api/external-auth/auto-added-scopes` | GET | Get auto-added scopes mapping | +| `/api/apps/{app_id}/external-auth/list` | GET | List all connectors | +| `/api/apps/{app_id}/external-auth/initiate` | POST | Start OAuth flow | +| `/api/apps/{app_id}/external-auth/status` | GET | Poll for OAuth completion | +| `/api/apps/{app_id}/external-auth/integrations/{type}/remove` | DELETE | Hard delete connector | + +### Subtasks + +2.1. **Auto-added scopes endpoint** (no auth required) + ```typescript + async getAutoAddedScopes(): Promise> + ``` + - Response is highly cacheable (data rarely changes) + - Consider caching in CLI session + +2.2. **List connectors** + ```typescript + async listConnectors(appId: string): Promise + ``` + +2.3. **Initiate OAuth** + ```typescript + async initiateOAuth(appId: string, request: { + integration_type: string; + scopes: string[]; + force_reconnect: boolean; + }): Promise<{ + redirect_url: string; + connection_id: string; + already_authorized: boolean; + }> + ``` + +2.4. **Poll status** + ```typescript + async getOAuthStatus(appId: string, params: { + integration_type: string; + connection_id: string; + }): Promise<{ status: 'ACTIVE' | 'FAILED' | 'PENDING' }> + ``` + +2.5. **Hard delete** + ```typescript + async removeConnector(appId: string, type: string): Promise + ``` + +### Implementation location +- Add to existing API client or create `src/api/external-auth.ts` + +--- + +## Task 3: Push Comparison Logic + +### Objective +Compare local connector definitions with upstream state and determine required actions. + +### Scope Comparison Logic + +```typescript +function scopesMatch( + localScopes: string[], + upstreamScopes: string[], + autoAddedScopes: string[] +): boolean { + const expected = new Set([...localScopes, ...autoAddedScopes]); + const upstream = new Set(upstreamScopes); + + if (expected.size !== upstream.size) return false; + for (const scope of expected) { + if (!upstream.has(scope)) return false; + } + return true; +} +``` + +### Comparison Matrix + +| Local File | Upstream State | Scopes Match? | Action | +|------------|----------------|---------------|--------| +| Exists | Not exists | N/A | Prompt auth URL + poll | +| Exists | `DISCONNECTED` | N/A | Hard delete + prompt auth URL + poll | +| Exists | `EXPIRED` | N/A | Hard delete + prompt auth URL + poll | +| Exists | `ACTIVE` | Yes | No-op | +| Exists | `ACTIVE` | No | Hard delete + prompt auth URL + poll | +| Not exists | Exists (any) | N/A | Hard delete upstream | + +### Subtasks + +3.1. **Fetch auto-added scopes** + - Call `/api/external-auth/auto-added-scopes` + - Cache for session duration + +3.2. **Calculate expected scopes** + ```typescript + expectedScopes = localScopes ∪ autoAddedScopes + ``` + +3.3. **Compare and determine actions** + - For each local connector: determine if auth needed + - For each upstream-only connector: mark for deletion + +3.4. **Implementation location** + - Create `src/resources/connectors/push.ts` + +--- + +## Task 4: OAuth Flow Handling + +### Objective +Handle interactive OAuth authentication with URL display and polling. + +### Flow + +1. Display auth URL to user (open in browser or show URL) +2. Poll `/status` endpoint every 2 seconds +3. Use same timeout as device login auth +4. Verify approved scopes after completion + +### Post-Auth Verification + +```typescript +function verifyApprovedScopes( + localScopes: string[], + approvedScopes: string[], + autoAddedScopes: string[] +): boolean { + const expected = new Set([...localScopes, ...autoAddedScopes]); + const approved = new Set(approvedScopes); + + if (expected.size !== approved.size) return false; + for (const scope of expected) { + if (!approved.has(scope)) return false; + } + return true; +} +``` + +### Subtasks + +4.1. **Auth URL display** + - Show URL to user + - Optionally auto-open in browser + +4.2. **Polling loop** + - 2 second interval + - Timeout matching device auth + - Handle ACTIVE, FAILED, PENDING states + +4.3. **Sequential auth** + - If multiple connectors need auth, process one at a time + - Interactive state machine + +4.4. **Edge cases** + | Case | Handling | + |------|----------| + | Partial consent | Show `SCOPE_MISMATCH` status | + | Different user | Show `DIFFERENT_USER` status with email | + | Auth timeout | Show `PENDING_AUTH` status | + | Auth failed | Show `AUTH_FAILED` status | + +4.5. **Why always delete first?** + - Apper's `/initiate` merges new scopes with existing (for AI use case) + - CLI needs declarative state (JSONC = desired scopes) + - Delete first ensures exact scopes, not union of old + new + +--- + +## Task 5: Push Command Integration + +### Objective +Wire connectors into the existing push command with summary output. + +### Summary Output Format + +``` +Connectors push summary: + - googlecalendar: active (3 scopes) + - slack: active (4 scopes, re-authed) + - linkedin: scope mismatch (requested 5, approved 3) + - notion: auth not completed + - hubspot: deleted (no local definition) + +Some connectors need attention: + - linkedin: Approved scopes differ from requested. Update connectors/linkedin.jsonc or run push again. + - notion: Authentication not completed. Run push to retry. +``` + +### Status States + +| Status | Color | Description | +|--------|-------|-------------| +| `ACTIVE` | green | Connector active, scopes match | +| `SCOPE_MISMATCH` | yellow | Active but approved ≠ requested | +| `PENDING_AUTH` | red | Auth URL shown but not completed | +| `AUTH_FAILED` | red | OAuth flow failed | +| `DELETED` | dim | Removed from upstream | +| `DIFFERENT_USER` | red | Another user already authorized | + +### Subtasks + +5.1. **Integrate with push command** + - Add connectors to resource types handled by push + - No pull support (push-only resource) + +5.2. **Summary output** + - Use `@clack/prompts` for logging + - Color statuses with chalk + +5.3. **Attention section** + - Show actionable messages for non-success states + +--- + +## File Structure (Proposed) + +``` +src/ +├── resources/ +│ └── connectors/ +│ ├── schema.ts # Zod schemas +│ ├── reader.ts # File reading +│ ├── push.ts # Push logic +│ └── index.ts # Exports +├── api/ +│ └── external-auth.ts # API client methods +└── commands/ + └── push.ts # Updated to include connectors +``` + +--- ## Dependencies -### Core CLI (bundled - zero runtime dependencies) -All dependencies are bundled into a single file at build time using tsdown. - -- **commander** - CLI framework for command parsing and help generation -- **@clack/prompts** - Beautiful, accessible prompts and UI components -- **chalk** - Terminal colors (Base44 brand color: #E86B3C) -- **json5** - JSONC/JSON5 config file parsing (supports comments and trailing commas) -- **zod** - Schema validation for API responses, config files, and inputs -- **ky** - HTTP client for API communication -- **ejs** - Template rendering for project scaffolding -- **globby** - File globbing for resource discovery -- **dotenv** - Environment variable loading - -### Development -- **typescript** - TypeScript compiler and type system -- **tsx** - TypeScript execution for development/watch mode -- **tsdown** - Bundler (powered by Rolldown) for zero-dependency distribution -- **vitest** - Testing framework -- **@types/node** - TypeScript definitions for Node.js +- Existing: `zod`, `@clack/prompts`, `chalk` +- Backend: Issue #3325 for `/api/external-auth/auto-added-scopes` endpoint + +--- + +## Testing Considerations +- Unit tests for scope comparison logic +- Unit tests for Zod schema validation +- Integration tests for push flow (mock API) +- Manual testing of OAuth flow (requires real OAuth providers) diff --git a/src/cli/commands/connectors/index.ts b/src/cli/commands/connectors/index.ts new file mode 100644 index 00000000..191915b9 --- /dev/null +++ b/src/cli/commands/connectors/index.ts @@ -0,0 +1,9 @@ +import { Command } from "commander"; +import type { CLIContext } from "@/cli/types.js"; +import { getConnectorsPushCommand } from "./push.js"; + +export function getConnectorsCommand(context: CLIContext): Command { + return new Command("connectors") + .description("Manage project connectors (OAuth integrations)") + .addCommand(getConnectorsPushCommand(context)); +} diff --git a/src/cli/commands/connectors/push.ts b/src/cli/commands/connectors/push.ts new file mode 100644 index 00000000..55f04e61 --- /dev/null +++ b/src/cli/commands/connectors/push.ts @@ -0,0 +1,161 @@ +import { confirm, isCancel, log } from "@clack/prompts"; +import chalk from "chalk"; +import { Command } from "commander"; +import type { CLIContext } from "@/cli/types.js"; +import { runCommand, runTask } from "@/cli/utils/index.js"; +import type { RunCommandResult } from "@/cli/utils/runCommand.js"; +import { readProjectConfig } from "@/core/index.js"; +import { + type ConnectorOAuthStatus, + type ConnectorSyncResult, + type IntegrationType, + pushConnectors, + runOAuthFlow, +} from "@/core/resources/connector/index.js"; + +type PendingOAuthResult = ConnectorSyncResult & { + redirectUrl: string; + connectionId: string; +}; + +function isPendingOAuth(r: ConnectorSyncResult): r is PendingOAuthResult { + return r.action === "needs_oauth" && !!r.redirectUrl && !!r.connectionId; +} + +function printSummary( + results: ConnectorSyncResult[], + oauthOutcomes: Map +): void { + const synced: IntegrationType[] = []; + const added: IntegrationType[] = []; + const removed: IntegrationType[] = []; + const failed: { type: IntegrationType; error?: string }[] = []; + + for (const r of results) { + const oauthStatus = oauthOutcomes.get(r.type); + + if (r.action === "synced") { + synced.push(r.type); + } else if (r.action === "removed") { + removed.push(r.type); + } else if (r.action === "error") { + failed.push({ type: r.type, error: r.error }); + } else if (r.action === "needs_oauth") { + if (oauthStatus === "ACTIVE") { + added.push(r.type); + } else if (oauthStatus === "PENDING") { + failed.push({ type: r.type, error: "authorization timed out" }); + } else if (oauthStatus === "FAILED") { + failed.push({ type: r.type, error: "authorization failed" }); + } else { + failed.push({ type: r.type, error: "needs authorization" }); + } + } + } + + log.info(""); + log.info(chalk.bold("Summary:")); + + if (synced.length > 0) { + log.info(chalk.green(` Synced: ${synced.join(", ")}`)); + } + if (added.length > 0) { + log.info(chalk.green(` Added: ${added.join(", ")}`)); + } + if (removed.length > 0) { + log.info(chalk.dim(` Removed: ${removed.join(", ")}`)); + } + for (const r of failed) { + log.info(chalk.red(` Failed: ${r.type}${r.error ? ` - ${r.error}` : ""}`)); + } +} + +async function pushConnectorsAction(): Promise { + const { connectors } = await readProjectConfig(); + + if (connectors.length === 0) { + log.info( + "No local connectors found - checking for remote connectors to remove" + ); + } else { + const connectorNames = connectors.map((c) => c.type).join(", "); + log.info( + `Found ${connectors.length} connectors to push: ${connectorNames}` + ); + } + + const { results } = await runTask( + "Pushing connectors to Base44", + async () => { + return await pushConnectors(connectors); + }, + { + successMessage: "Connectors pushed", + errorMessage: "Failed to push connectors", + } + ); + + const oauthOutcomes = new Map(); + const needsOAuth = results.filter(isPendingOAuth); + let outroMessage = "Connectors pushed to Base44"; + + if (needsOAuth.length > 0) { + log.info(""); + log.info( + chalk.yellow( + `${needsOAuth.length} connector(s) require authorization in your browser:` + ) + ); + for (const connector of needsOAuth) { + log.info(` ${connector.type}: ${chalk.dim(connector.redirectUrl)}`); + } + + const pending = needsOAuth.map((c) => c.type).join(", "); + + if (process.env.CI) { + outroMessage = `Skipped OAuth in CI. Pending: ${pending}. Run 'base44 connectors push' locally to authorize.`; + } else { + const shouldAuth = await confirm({ + message: "Open browser to authorize now?", + }); + + if (isCancel(shouldAuth) || !shouldAuth) { + outroMessage = `Authorization skipped. Pending: ${pending}. Run 'base44 connectors push' again to complete.`; + } else { + for (const connector of needsOAuth) { + log.info(`\nOpening browser for ${connector.type}...`); + + const oauthResult = await runTask( + `Waiting for ${connector.type} authorization...`, + async () => { + return await runOAuthFlow({ + type: connector.type, + redirectUrl: connector.redirectUrl, + connectionId: connector.connectionId, + }); + }, + { + successMessage: `${connector.type} authorization complete`, + errorMessage: `${connector.type} authorization failed`, + } + ); + + oauthOutcomes.set(connector.type, oauthResult.status); + } + } + } + } + + printSummary(results, oauthOutcomes); + return { outroMessage }; +} + +export function getConnectorsPushCommand(context: CLIContext): Command { + return new Command("push") + .description( + "Push local connectors to Base44 (syncs scopes and removes unlisted)" + ) + .action(async () => { + await runCommand(pushConnectorsAction, { requireAuth: true }, context); + }); +} diff --git a/src/cli/program.ts b/src/cli/program.ts index 15e4f1d7..80505dd8 100644 --- a/src/cli/program.ts +++ b/src/cli/program.ts @@ -3,6 +3,7 @@ import { getAgentsCommand } from "@/cli/commands/agents/index.js"; import { getLoginCommand } from "@/cli/commands/auth/login.js"; import { getLogoutCommand } from "@/cli/commands/auth/logout.js"; import { getWhoamiCommand } from "@/cli/commands/auth/whoami.js"; +import { getConnectorsCommand } from "@/cli/commands/connectors/index.js"; import { getDashboardCommand } from "@/cli/commands/dashboard/index.js"; import { getEntitiesPushCommand } from "@/cli/commands/entities/push.js"; import { getFunctionsDeployCommand } from "@/cli/commands/functions/deploy.js"; @@ -45,6 +46,9 @@ export function createProgram(context: CLIContext): Command { // Register agents commands program.addCommand(getAgentsCommand(context)); + // Register connectors commands + program.addCommand(getConnectorsCommand(context)); + // Register functions commands program.addCommand(getFunctionsDeployCommand(context)); diff --git a/src/core/project/config.ts b/src/core/project/config.ts index 36738a06..5dc1abd7 100644 --- a/src/core/project/config.ts +++ b/src/core/project/config.ts @@ -5,6 +5,7 @@ import { ConfigNotFoundError, SchemaValidationError } from "@/core/errors.js"; import { ProjectConfigSchema } from "@/core/project/schema.js"; import type { ProjectData, ProjectRoot } from "@/core/project/types.js"; import { agentResource } from "@/core/resources/agent/index.js"; +import { connectorResource } from "@/core/resources/connector/index.js"; import { entityResource } from "@/core/resources/entity/index.js"; import { functionResource } from "@/core/resources/function/index.js"; import { readJsonFile } from "@/core/utils/fs.js"; @@ -91,10 +92,11 @@ export async function readProjectConfig( const project = result.data; const configDir = dirname(configPath); - const [entities, functions, agents] = await Promise.all([ + const [entities, functions, agents, connectors] = await Promise.all([ entityResource.readAll(join(configDir, project.entitiesDir)), functionResource.readAll(join(configDir, project.functionsDir)), agentResource.readAll(join(configDir, project.agentsDir)), + connectorResource.readAll(join(configDir, project.connectorsDir)), ]); return { @@ -102,5 +104,6 @@ export async function readProjectConfig( entities, functions, agents, + connectors, }; } diff --git a/src/core/project/schema.ts b/src/core/project/schema.ts index c9a00c24..ed1927bc 100644 --- a/src/core/project/schema.ts +++ b/src/core/project/schema.ts @@ -32,6 +32,7 @@ export const ProjectConfigSchema = z.object({ entitiesDir: z.string().optional().default("entities"), functionsDir: z.string().optional().default("functions"), agentsDir: z.string().optional().default("agents"), + connectorsDir: z.string().optional().default("connectors"), }); export type SiteConfig = z.infer; diff --git a/src/core/project/types.ts b/src/core/project/types.ts index c910aa47..f1e8fa5c 100644 --- a/src/core/project/types.ts +++ b/src/core/project/types.ts @@ -1,5 +1,6 @@ import type { ProjectConfig } from "@/core/project/schema.js"; import type { AgentConfig } from "@/core/resources/agent/index.js"; +import type { ConnectorResource } from "@/core/resources/connector/index.js"; import type { Entity } from "@/core/resources/entity/index.js"; import type { BackendFunction } from "@/core/resources/function/index.js"; @@ -18,4 +19,5 @@ export interface ProjectData { entities: Entity[]; functions: BackendFunction[]; agents: AgentConfig[]; + connectors: ConnectorResource[]; } diff --git a/src/core/resources/connector/api.ts b/src/core/resources/connector/api.ts new file mode 100644 index 00000000..a3dc2ad9 --- /dev/null +++ b/src/core/resources/connector/api.ts @@ -0,0 +1,130 @@ +import type { KyResponse } from "ky"; +import { getAppClient } from "@/core/clients/index.js"; +import { ApiError, SchemaValidationError } from "@/core/errors.js"; +import type { + IntegrationType, + ListConnectorsResponse, + OAuthStatusResponse, + RemoveConnectorResponse, + SetConnectorResponse, +} from "./schema.js"; +import { + ListConnectorsResponseSchema, + OAuthStatusResponseSchema, + RemoveConnectorResponseSchema, + SetConnectorResponseSchema, +} from "./schema.js"; + +/** + * List all connectors for the current app. + * GET /api/apps/{app_id}/external-auth/list + */ +export async function listConnectors(): Promise { + const appClient = getAppClient(); + + let response: KyResponse; + try { + response = await appClient.get("external-auth/list"); + } catch (error) { + throw await ApiError.fromHttpError(error, "listing connectors"); + } + + const result = ListConnectorsResponseSchema.safeParse(await response.json()); + + if (!result.success) { + throw new SchemaValidationError( + "Invalid response from server", + result.error + ); + } + + return result.data; +} + +export async function setConnector( + integrationType: IntegrationType, + scopes: string[] +): Promise { + const appClient = getAppClient(); + + let response: KyResponse; + try { + response = await appClient.put( + `external-auth/integrations/${integrationType}`, + { + json: { + scopes, + }, + } + ); + } catch (error) { + throw await ApiError.fromHttpError(error, "setting connector"); + } + + const result = SetConnectorResponseSchema.safeParse(await response.json()); + + if (!result.success) { + throw new SchemaValidationError( + "Invalid response from server", + result.error + ); + } + + return result.data; +} + +export async function getOAuthStatus( + integrationType: IntegrationType, + connectionId: string +): Promise { + const appClient = getAppClient(); + + let response: KyResponse; + try { + response = await appClient.get("external-auth/status", { + searchParams: { + integration_type: integrationType, + connection_id: connectionId, + }, + }); + } catch (error) { + throw await ApiError.fromHttpError(error, "checking OAuth status"); + } + + const result = OAuthStatusResponseSchema.safeParse(await response.json()); + + if (!result.success) { + throw new SchemaValidationError( + "Invalid response from server", + result.error + ); + } + + return result.data; +} + +export async function removeConnector( + integrationType: IntegrationType +): Promise { + const appClient = getAppClient(); + + let response: KyResponse; + try { + response = await appClient.delete( + `external-auth/integrations/${integrationType}/remove` + ); + } catch (error) { + throw await ApiError.fromHttpError(error, "removing connector"); + } + + const result = RemoveConnectorResponseSchema.safeParse(await response.json()); + + if (!result.success) { + throw new SchemaValidationError( + "Invalid response from server", + result.error + ); + } + + return result.data; +} diff --git a/src/core/resources/connector/config.ts b/src/core/resources/connector/config.ts new file mode 100644 index 00000000..78cdee84 --- /dev/null +++ b/src/core/resources/connector/config.ts @@ -0,0 +1,54 @@ +import { globby } from "globby"; +import { SchemaValidationError } from "@/core/errors.js"; +import { CONFIG_FILE_EXTENSION_GLOB } from "../../consts.js"; +import { pathExists, readJsonFile } from "../../utils/fs.js"; +import type { ConnectorResource } from "./schema.js"; +import { ConnectorResourceSchema } from "./schema.js"; + +async function readConnectorFile( + connectorPath: string +): Promise { + const parsed = await readJsonFile(connectorPath); + const result = ConnectorResourceSchema.safeParse(parsed); + + if (!result.success) { + throw new SchemaValidationError( + "Invalid connector file", + result.error, + connectorPath + ); + } + + return result.data; +} + +/** + * Read all connector files from a directory. + * Returns an empty array if the directory doesn't exist. + */ +export async function readAllConnectors( + connectorsDir: string +): Promise { + if (!(await pathExists(connectorsDir))) { + return []; + } + + const files = await globby(`*.${CONFIG_FILE_EXTENSION_GLOB}`, { + cwd: connectorsDir, + absolute: true, + }); + + const connectors = await Promise.all( + files.map((filePath) => readConnectorFile(filePath)) + ); + + const types = new Set(); + for (const connector of connectors) { + if (types.has(connector.type)) { + throw new Error(`Duplicate connector type "${connector.type}"`); + } + types.add(connector.type); + } + + return connectors; +} diff --git a/src/core/resources/connector/index.ts b/src/core/resources/connector/index.ts new file mode 100644 index 00000000..417096fc --- /dev/null +++ b/src/core/resources/connector/index.ts @@ -0,0 +1,6 @@ +export * from "./api.js"; +export * from "./config.js"; +export * from "./oauth.js"; +export * from "./push.js"; +export * from "./resource.js"; +export * from "./schema.js"; diff --git a/src/core/resources/connector/oauth.ts b/src/core/resources/connector/oauth.ts new file mode 100644 index 00000000..6efd2d37 --- /dev/null +++ b/src/core/resources/connector/oauth.ts @@ -0,0 +1,46 @@ +import open from "open"; +import pWaitFor, { TimeoutError } from "p-wait-for"; +import { getOAuthStatus } from "./api.js"; +import type { ConnectorOAuthStatus, IntegrationType } from "./schema.js"; + +const POLL_INTERVAL_MS = 2000; +const POLL_TIMEOUT_MS = 2 * 60 * 1000; // 2 minutes + +export interface OAuthFlowParams { + type: IntegrationType; + redirectUrl: string; + connectionId: string; +} + +export interface OAuthFlowResult { + type: IntegrationType; + status: ConnectorOAuthStatus; +} + +export async function runOAuthFlow( + params: OAuthFlowParams +): Promise { + await open(params.redirectUrl); + + let finalStatus: ConnectorOAuthStatus = "PENDING"; + + await pWaitFor( + async () => { + const response = await getOAuthStatus(params.type, params.connectionId); + finalStatus = response.status; + return response.status !== "PENDING"; + }, + { + interval: POLL_INTERVAL_MS, + timeout: POLL_TIMEOUT_MS, + } + ).catch((err) => { + if (err instanceof TimeoutError) { + finalStatus = "PENDING"; + } else { + throw err; + } + }); + + return { type: params.type, status: finalStatus }; +} diff --git a/src/core/resources/connector/push.ts b/src/core/resources/connector/push.ts new file mode 100644 index 00000000..f386447c --- /dev/null +++ b/src/core/resources/connector/push.ts @@ -0,0 +1,89 @@ +import { listConnectors, removeConnector, setConnector } from "./api.js"; +import type { + ConnectorResource, + IntegrationType, + SetConnectorResponse, +} from "./schema.js"; + +export interface ConnectorSyncResult { + type: IntegrationType; + action: "synced" | "removed" | "needs_oauth" | "error"; + redirectUrl?: string; + connectionId?: string; + error?: string; +} + +export interface PushConnectorsResponse { + results: ConnectorSyncResult[]; +} + +export async function pushConnectors( + connectors: ConnectorResource[] +): Promise { + const results: ConnectorSyncResult[] = []; + const upstream = await listConnectors(); + const localTypes = new Set(connectors.map((c) => c.type)); + + for (const connector of connectors) { + try { + const response = await setConnector(connector.type, connector.scopes); + results.push(setResponseToResult(connector.type, response)); + } catch (err) { + results.push({ + type: connector.type, + action: "error", + error: err instanceof Error ? err.message : String(err), + }); + } + } + + for (const upstreamConnector of upstream.integrations) { + if (!localTypes.has(upstreamConnector.integration_type)) { + try { + await removeConnector(upstreamConnector.integration_type); + results.push({ + type: upstreamConnector.integration_type, + action: "removed", + }); + } catch (err) { + results.push({ + type: upstreamConnector.integration_type, + action: "error", + error: err instanceof Error ? err.message : String(err), + }); + } + } + } + + return { results }; +} + +function setResponseToResult( + type: IntegrationType, + response: SetConnectorResponse +): ConnectorSyncResult { + if (response.error === "different_user") { + return { + type, + action: "error", + error: + response.error_message || + `Already connected by ${response.other_user_email}`, + }; + } + + if (response.already_authorized) { + return { type, action: "synced" }; + } + + if (response.redirect_url) { + return { + type, + action: "needs_oauth", + redirectUrl: response.redirect_url, + connectionId: response.connection_id ?? undefined, + }; + } + + return { type, action: "synced" }; +} diff --git a/src/core/resources/connector/resource.ts b/src/core/resources/connector/resource.ts new file mode 100644 index 00000000..14f57e1a --- /dev/null +++ b/src/core/resources/connector/resource.ts @@ -0,0 +1,9 @@ +import type { Resource } from "../types.js"; +import { readAllConnectors } from "./config.js"; +import { pushConnectors } from "./push.js"; +import type { ConnectorResource } from "./schema.js"; + +export const connectorResource: Resource = { + readAll: readAllConnectors, + push: pushConnectors, +}; diff --git a/src/core/resources/connector/schema.ts b/src/core/resources/connector/schema.ts new file mode 100644 index 00000000..ff5be687 --- /dev/null +++ b/src/core/resources/connector/schema.ts @@ -0,0 +1,166 @@ +import { z } from "zod"; + +/** Google Calendar - Scopes: https://developers.google.com/identity/protocols/oauth2/scopes#calendar */ +export const GoogleCalendarConnectorSchema = z.object({ + type: z.literal("googlecalendar"), + scopes: z.array(z.string()).default([]), +}); + +/** Google Drive - Scopes: https://developers.google.com/identity/protocols/oauth2/scopes#drive */ +export const GoogleDriveConnectorSchema = z.object({ + type: z.literal("googledrive"), + scopes: z.array(z.string()).default([]), +}); + +/** Gmail - Scopes: https://developers.google.com/identity/protocols/oauth2/scopes#gmail */ +export const GmailConnectorSchema = z.object({ + type: z.literal("gmail"), + scopes: z.array(z.string()).default([]), +}); + +/** Google Sheets - Scopes: https://developers.google.com/identity/protocols/oauth2/scopes#sheets */ +export const GoogleSheetsConnectorSchema = z.object({ + type: z.literal("googlesheets"), + scopes: z.array(z.string()).default([]), +}); + +/** Google Docs - Scopes: https://developers.google.com/identity/protocols/oauth2/scopes#docs */ +export const GoogleDocsConnectorSchema = z.object({ + type: z.literal("googledocs"), + scopes: z.array(z.string()).default([]), +}); + +/** Google Slides - Scopes: https://developers.google.com/identity/protocols/oauth2/scopes#slides */ +export const GoogleSlidesConnectorSchema = z.object({ + type: z.literal("googleslides"), + scopes: z.array(z.string()).default([]), +}); + +/** Slack - Scopes: https://api.slack.com/scopes */ +export const SlackConnectorSchema = z.object({ + type: z.literal("slack"), + scopes: z.array(z.string()).default([]), +}); + +/** Notion - Scopes are preauthorized, no need to request them explicitly (values will be ignored) */ +export const NotionConnectorSchema = z.object({ + type: z.literal("notion"), + scopes: z.array(z.string()).default([]), +}); + +/** Salesforce - Scopes: https://developer.salesforce.com/docs/platform/mobile-sdk/guide/oauth-scope-parameter-values.html */ +export const SalesforceConnectorSchema = z.object({ + type: z.literal("salesforce"), + scopes: z.array(z.string()).default([]), +}); + +/** HubSpot - Scopes: https://developers.hubspot.com/docs/api/scopes */ +export const HubspotConnectorSchema = z.object({ + type: z.literal("hubspot"), + scopes: z.array(z.string()).default([]), +}); + +/** LinkedIn - Scopes: https://learn.microsoft.com/en-us/linkedin/shared/authentication/authorization-code-flow */ +export const LinkedInConnectorSchema = z.object({ + type: z.literal("linkedin"), + scopes: z.array(z.string()).default([]), +}); + +/** TikTok - Scopes: https://developers.tiktok.com/doc/tiktok-api-scopes */ +export const TikTokConnectorSchema = z.object({ + type: z.literal("tiktok"), + scopes: z.array(z.string()).default([]), +}); + +export const ConnectorResourceSchema = z.discriminatedUnion("type", [ + GoogleCalendarConnectorSchema, + GoogleDriveConnectorSchema, + GmailConnectorSchema, + GoogleSheetsConnectorSchema, + GoogleDocsConnectorSchema, + GoogleSlidesConnectorSchema, + SlackConnectorSchema, + NotionConnectorSchema, + SalesforceConnectorSchema, + HubspotConnectorSchema, + LinkedInConnectorSchema, + TikTokConnectorSchema, +]); + +export type ConnectorResource = z.infer; + +export const IntegrationTypeSchema = z.enum([ + "googlecalendar", + "googledrive", + "gmail", + "googlesheets", + "googledocs", + "googleslides", + "slack", + "notion", + "salesforce", + "hubspot", + "linkedin", + "tiktok", +]); + +export type IntegrationType = z.infer; + +export const ConnectorStatusSchema = z.enum([ + "active", + "disconnected", + "expired", +]); + +export type ConnectorStatus = z.infer; + +export const UpstreamConnectorSchema = z.object({ + integration_type: IntegrationTypeSchema, + status: ConnectorStatusSchema, + scopes: z.array(z.string()), + user_email: z.string().optional(), +}); + +export type UpstreamConnector = z.infer; + +export const ListConnectorsResponseSchema = z.object({ + integrations: z.array(UpstreamConnectorSchema), +}); + +export type ListConnectorsResponse = z.infer< + typeof ListConnectorsResponseSchema +>; + +export const SetConnectorResponseSchema = z.object({ + redirect_url: z.string().nullable(), + connection_id: z.string().nullable(), + already_authorized: z.boolean(), + error: z.string().nullable().optional(), + error_message: z.string().nullable().optional(), + other_user_email: z.string().nullable().optional(), +}); + +export type SetConnectorResponse = z.infer; + +export const ConnectorOAuthStatusSchema = z.enum([ + "ACTIVE", + "FAILED", + "PENDING", +]); + +export type ConnectorOAuthStatus = z.infer; + +export const OAuthStatusResponseSchema = z.object({ + status: ConnectorOAuthStatusSchema, +}); + +export type OAuthStatusResponse = z.infer; + +export const RemoveConnectorResponseSchema = z.object({ + status: z.literal("removed"), + integration_type: IntegrationTypeSchema, +}); + +export type RemoveConnectorResponse = z.infer< + typeof RemoveConnectorResponseSchema +>; diff --git a/src/core/resources/index.ts b/src/core/resources/index.ts index 191bbdb9..93ddcc26 100644 --- a/src/core/resources/index.ts +++ b/src/core/resources/index.ts @@ -1,4 +1,5 @@ export * from "./agent/index.js"; +export * from "./connector/index.js"; export * from "./entity/index.js"; export * from "./function/index.js"; export type { Resource } from "./types.js"; diff --git a/tests/cli/connectors_push.spec.ts b/tests/cli/connectors_push.spec.ts new file mode 100644 index 00000000..33fa391b --- /dev/null +++ b/tests/cli/connectors_push.spec.ts @@ -0,0 +1,122 @@ +import { describe, it } from "vitest"; +import { fixture, setupCLITests } from "./testkit/index.js"; + +describe("connectors push command", () => { + const t = setupCLITests(); + + it("shows message when no local connectors found", async () => { + await t.givenLoggedInWithProject(fixture("basic")); + t.api.mockConnectorsList({ integrations: [] }); + + const result = await t.run("connectors", "push"); + + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("No local connectors found"); + }); + + it("fails when not in a project directory", async () => { + await t.givenLoggedIn({ email: "test@example.com", name: "Test User" }); + + const result = await t.run("connectors", "push"); + + t.expectResult(result).toFail(); + t.expectResult(result).toContain("No Base44 project found"); + }); + + it("finds and lists connectors in project", async () => { + await t.givenLoggedInWithProject(fixture("with-connectors")); + t.api.mockConnectorsList({ integrations: [] }); + t.api.mockConnectorSet({ + redirect_url: null, + connection_id: null, + already_authorized: true, + }); + + const result = await t.run("connectors", "push"); + + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("Found 3 connectors to push"); + }); + + it("displays synced connectors with checkmark", async () => { + await t.givenLoggedInWithProject(fixture("with-connectors")); + t.api.mockConnectorsList({ integrations: [] }); + t.api.mockConnectorSet({ + redirect_url: null, + connection_id: null, + already_authorized: true, + }); + + const result = await t.run("connectors", "push"); + + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("googlecalendar"); + t.expectResult(result).toContain("slack"); + t.expectResult(result).toContain("notion"); + }); + + it("displays removed connectors", async () => { + await t.givenLoggedInWithProject(fixture("basic")); + t.api.mockConnectorsList({ + integrations: [ + { integration_type: "slack", status: "ACTIVE", scopes: ["chat:write"] }, + ], + }); + t.api.mockConnectorRemove({ status: "removed", integration_type: "slack" }); + + const result = await t.run("connectors", "push"); + + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("slack"); + t.expectResult(result).toContain("Removed:"); + }); + + it("displays error when sync fails", async () => { + await t.givenLoggedInWithProject(fixture("with-connectors")); + t.api.mockConnectorsList({ integrations: [] }); + t.api.mockConnectorSetError({ + status: 500, + body: { error: "Server error" }, + }); + + const result = await t.run("connectors", "push"); + + // Errors are handled per-connector, command still succeeds + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("googlecalendar"); + }); + + it("shows needs authorization when redirect_url is returned", async () => { + await t.givenLoggedInWithProject(fixture("with-connectors")); + t.api.mockConnectorsList({ integrations: [] }); + t.api.mockConnectorSet({ + redirect_url: "https://accounts.google.com/oauth", + connection_id: "conn_123", + already_authorized: false, + }); + + const result = await t.run("connectors", "push"); + + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("needs authorization"); + t.expectResult(result).toContain("Skipped OAuth in CI"); + }); + + it("shows error for different_user response", async () => { + await t.givenLoggedInWithProject(fixture("with-connectors")); + t.api.mockConnectorsList({ integrations: [] }); + t.api.mockConnectorSet({ + redirect_url: null, + connection_id: null, + already_authorized: false, + error: "different_user", + error_message: "Already connected by another user", + other_user_email: "other@example.com", + }); + + const result = await t.run("connectors", "push"); + + t.expectResult(result).toSucceed(); + t.expectResult(result).toContain("Already connected by another user"); + }); +}); diff --git a/tests/cli/testkit/Base44APIMock.ts b/tests/cli/testkit/Base44APIMock.ts index c1e1d4ec..8776836c 100644 --- a/tests/cli/testkit/Base44APIMock.ts +++ b/tests/cli/testkit/Base44APIMock.ts @@ -57,6 +57,33 @@ export interface AgentsFetchResponse { total: number; } +export interface ConnectorsListResponse { + integrations: Array<{ + integration_type: string; + status: string; + scopes: string[]; + user_email?: string; + }>; +} + +export interface ConnectorSetResponse { + redirect_url: string | null; + connection_id: string | null; + already_authorized: boolean; + error?: "different_user"; + error_message?: string; + other_user_email?: string; +} + +export interface ConnectorOAuthStatusResponse { + status: "ACTIVE" | "FAILED" | "PENDING"; +} + +export interface ConnectorRemoveResponse { + status: "removed"; + integration_type: string; +} + export interface CreateAppResponse { id: string; name: string; @@ -182,6 +209,50 @@ export class Base44APIMock { return this; } + // ─── CONNECTOR ENDPOINTS ────────────────────────────────── + + /** Mock GET /api/apps/{appId}/external-auth/list - List connectors */ + mockConnectorsList(response: ConnectorsListResponse): this { + this.handlers.push( + http.get(`${BASE_URL}/api/apps/${this.appId}/external-auth/list`, () => + HttpResponse.json(response) + ) + ); + return this; + } + + /** Mock PUT /api/apps/{appId}/external-auth/integrations/{type} - Set connector */ + mockConnectorSet(response: ConnectorSetResponse): this { + this.handlers.push( + http.put( + `${BASE_URL}/api/apps/${this.appId}/external-auth/integrations/:type`, + () => HttpResponse.json(response) + ) + ); + return this; + } + + /** Mock GET /api/apps/{appId}/external-auth/status - Get OAuth status */ + mockConnectorOAuthStatus(response: ConnectorOAuthStatusResponse): this { + this.handlers.push( + http.get(`${BASE_URL}/api/apps/${this.appId}/external-auth/status`, () => + HttpResponse.json(response) + ) + ); + return this; + } + + /** Mock DELETE /api/apps/{appId}/external-auth/integrations/{type}/remove */ + mockConnectorRemove(response: ConnectorRemoveResponse): this { + this.handlers.push( + http.delete( + `${BASE_URL}/api/apps/${this.appId}/external-auth/integrations/:type/remove`, + () => HttpResponse.json(response) + ) + ); + return this; + } + // ─── GENERAL ENDPOINTS ───────────────────────────────────── /** Mock POST /api/apps - Create new app */ @@ -273,6 +344,24 @@ export class Base44APIMock { return this.mockError("get", "/oauth/userinfo", error); } + /** Mock connectors list to return an error */ + mockConnectorsListError(error: ErrorResponse): this { + return this.mockError( + "get", + `/api/apps/${this.appId}/external-auth/list`, + error + ); + } + + /** Mock connector set to return an error */ + mockConnectorSetError(error: ErrorResponse): this { + return this.mockError( + "put", + `/api/apps/${this.appId}/external-auth/integrations/:type`, + error + ); + } + // ─── INTERNAL ────────────────────────────────────────────── /** Apply all registered handlers to MSW (called by CLITestkit.run()) */ diff --git a/tests/core/connectors.spec.ts b/tests/core/connectors.spec.ts new file mode 100644 index 00000000..8bf6ea9b --- /dev/null +++ b/tests/core/connectors.spec.ts @@ -0,0 +1,401 @@ +import { resolve } from "node:path"; +import { beforeEach, describe, expect, it, vi } from "vitest"; +import * as api from "../../src/core/resources/connector/api.js"; +import { readAllConnectors } from "../../src/core/resources/connector/config.js"; +import { + type OAuthFlowParams, + runOAuthFlow, +} from "../../src/core/resources/connector/oauth.js"; +import { pushConnectors } from "../../src/core/resources/connector/push.js"; +import { + type ConnectorResource, + ConnectorResourceSchema, + IntegrationTypeSchema, +} from "../../src/core/resources/connector/schema.js"; + +vi.mock("../../src/core/resources/connector/api.js"); +vi.mock("open", () => ({ default: vi.fn() })); + +const FIXTURES_DIR = resolve(__dirname, "../fixtures"); + +describe("IntegrationTypeSchema", () => { + it("accepts valid integration types", () => { + const validTypes = [ + "googlecalendar", + "googledrive", + "gmail", + "googlesheets", + "googledocs", + "googleslides", + "slack", + "notion", + "salesforce", + "hubspot", + "linkedin", + "tiktok", + ]; + + for (const type of validTypes) { + expect(IntegrationTypeSchema.safeParse(type).success).toBe(true); + } + }); + + it("rejects invalid integration types", () => { + const invalidTypes = ["invalid", "google", "facebook", "twitter", ""]; + + for (const type of invalidTypes) { + expect(IntegrationTypeSchema.safeParse(type).success).toBe(false); + } + }); +}); + +describe("ConnectorResourceSchema", () => { + it("accepts valid connector with scopes", () => { + const connector = { + type: "googlecalendar", + scopes: [ + "https://www.googleapis.com/auth/calendar.readonly", + "https://www.googleapis.com/auth/calendar.events", + ], + }; + + const result = ConnectorResourceSchema.safeParse(connector); + expect(result.success).toBe(true); + if (result.success) { + expect(result.data.type).toBe("googlecalendar"); + expect(result.data.scopes).toHaveLength(2); + } + }); + + it("accepts valid connector with empty scopes", () => { + const connector = { + type: "notion", + scopes: [], + }; + + const result = ConnectorResourceSchema.safeParse(connector); + expect(result.success).toBe(true); + if (result.success) { + expect(result.data.scopes).toEqual([]); + } + }); + + it("defaults scopes to empty array if not provided", () => { + const connector = { + type: "slack", + }; + + const result = ConnectorResourceSchema.safeParse(connector); + expect(result.success).toBe(true); + if (result.success) { + expect(result.data.scopes).toEqual([]); + } + }); + + it("rejects connector with invalid type", () => { + const connector = { + type: "invalid", + scopes: [], + }; + + const result = ConnectorResourceSchema.safeParse(connector); + expect(result.success).toBe(false); + }); + + it("rejects connector without type", () => { + const connector = { + scopes: [], + }; + + const result = ConnectorResourceSchema.safeParse(connector); + expect(result.success).toBe(false); + }); +}); + +describe("readAllConnectors", () => { + it("returns empty array for non-existent directory", async () => { + const connectors = await readAllConnectors("/non/existent/path"); + expect(connectors).toEqual([]); + }); + + it("reads connectors from directory", async () => { + const connectorsDir = resolve(FIXTURES_DIR, "with-connectors/connectors"); + const connectors = await readAllConnectors(connectorsDir); + + expect(connectors).toHaveLength(3); + + const types = connectors.map((c) => c.type).sort(); + expect(types).toEqual(["googlecalendar", "notion", "slack"]); + + const googleCalendar = connectors.find((c) => c.type === "googlecalendar"); + expect(googleCalendar?.scopes).toEqual([ + "https://www.googleapis.com/auth/calendar.readonly", + "https://www.googleapis.com/auth/calendar.events", + ]); + + const notion = connectors.find((c) => c.type === "notion"); + expect(notion?.scopes).toEqual([]); + }); + + it("throws error for invalid connector type", async () => { + const connectorsDir = resolve(FIXTURES_DIR, "invalid-connector/connectors"); + + await expect(readAllConnectors(connectorsDir)).rejects.toThrow( + "Invalid connector file" + ); + }); +}); + +const mockListConnectors = vi.mocked(api.listConnectors); +const mockSetConnector = vi.mocked(api.setConnector); +const mockRemoveConnector = vi.mocked(api.removeConnector); + +describe("pushConnectors", () => { + beforeEach(() => { + vi.resetAllMocks(); + mockListConnectors.mockResolvedValue({ integrations: [] }); + }); + + it("returns empty results when no local or upstream connectors", async () => { + const result = await pushConnectors([]); + expect(result.results).toEqual([]); + expect(mockListConnectors).toHaveBeenCalledOnce(); + }); + + it("syncs local connectors", async () => { + const local: ConnectorResource[] = [ + { type: "gmail", scopes: ["https://mail.google.com/"] }, + ]; + mockSetConnector.mockResolvedValue({ + redirect_url: null, + connection_id: null, + already_authorized: true, + }); + + const result = await pushConnectors(local); + + expect(mockSetConnector).toHaveBeenCalledWith("gmail", [ + "https://mail.google.com/", + ]); + expect(result.results).toEqual([{ type: "gmail", action: "synced" }]); + }); + + it("removes upstream-only connectors", async () => { + mockListConnectors.mockResolvedValue({ + integrations: [ + { integration_type: "slack", status: "ACTIVE", scopes: ["chat:write"] }, + ], + }); + mockRemoveConnector.mockResolvedValue({ + status: "removed", + integration_type: "slack", + }); + + const result = await pushConnectors([]); + + expect(mockRemoveConnector).toHaveBeenCalledWith("slack"); + expect(result.results).toEqual([{ type: "slack", action: "removed" }]); + }); + + it("syncs local and removes upstream-only", async () => { + const local: ConnectorResource[] = [ + { type: "gmail", scopes: ["https://mail.google.com/"] }, + ]; + mockListConnectors.mockResolvedValue({ + integrations: [ + { integration_type: "slack", status: "ACTIVE", scopes: ["chat:write"] }, + ], + }); + mockSetConnector.mockResolvedValue({ + redirect_url: null, + connection_id: null, + already_authorized: true, + }); + mockRemoveConnector.mockResolvedValue({ + status: "removed", + integration_type: "slack", + }); + + const result = await pushConnectors(local); + + expect(mockSetConnector).toHaveBeenCalledWith("gmail", [ + "https://mail.google.com/", + ]); + expect(mockRemoveConnector).toHaveBeenCalledWith("slack"); + expect(result.results).toEqual([ + { type: "gmail", action: "synced" }, + { type: "slack", action: "removed" }, + ]); + }); + + it("does not remove connectors that exist locally", async () => { + const local: ConnectorResource[] = [ + { type: "gmail", scopes: ["https://mail.google.com/"] }, + ]; + mockListConnectors.mockResolvedValue({ + integrations: [ + { + integration_type: "gmail", + status: "ACTIVE", + scopes: ["https://mail.google.com/"], + }, + ], + }); + mockSetConnector.mockResolvedValue({ + redirect_url: null, + connection_id: null, + already_authorized: true, + }); + + const result = await pushConnectors(local); + + expect(mockRemoveConnector).not.toHaveBeenCalled(); + expect(result.results).toEqual([{ type: "gmail", action: "synced" }]); + }); + + it("returns needs_oauth when redirect_url is present", async () => { + const local: ConnectorResource[] = [ + { type: "gmail", scopes: ["https://mail.google.com/"] }, + ]; + mockSetConnector.mockResolvedValue({ + redirect_url: "https://accounts.google.com/oauth", + connection_id: "conn_123", + already_authorized: false, + }); + + const result = await pushConnectors(local); + + expect(result.results).toEqual([ + { + type: "gmail", + action: "needs_oauth", + redirectUrl: "https://accounts.google.com/oauth", + connectionId: "conn_123", + }, + ]); + }); + + it("returns error for different_user response", async () => { + const local: ConnectorResource[] = [ + { type: "gmail", scopes: ["https://mail.google.com/"] }, + ]; + mockSetConnector.mockResolvedValue({ + redirect_url: null, + connection_id: null, + already_authorized: false, + error: "different_user", + error_message: "Already connected by another user", + other_user_email: "other@example.com", + }); + + const result = await pushConnectors(local); + + expect(result.results).toEqual([ + { + type: "gmail", + action: "error", + error: "Already connected by another user", + }, + ]); + }); + + it("handles sync errors gracefully", async () => { + const local: ConnectorResource[] = [ + { type: "gmail", scopes: ["https://mail.google.com/"] }, + ]; + mockSetConnector.mockRejectedValue(new Error("Network error")); + + const result = await pushConnectors(local); + + expect(result.results).toEqual([ + { type: "gmail", action: "error", error: "Network error" }, + ]); + }); + + it("handles remove errors gracefully", async () => { + mockListConnectors.mockResolvedValue({ + integrations: [ + { integration_type: "slack", status: "ACTIVE", scopes: ["chat:write"] }, + ], + }); + mockRemoveConnector.mockRejectedValue(new Error("Remove failed")); + + const result = await pushConnectors([]); + + expect(result.results).toEqual([ + { type: "slack", action: "error", error: "Remove failed" }, + ]); + }); + + it("processes multiple local connectors", async () => { + const local: ConnectorResource[] = [ + { type: "gmail", scopes: ["https://mail.google.com/"] }, + { type: "slack", scopes: ["chat:write"] }, + ]; + mockSetConnector.mockResolvedValue({ + redirect_url: null, + connection_id: null, + already_authorized: true, + }); + + const result = await pushConnectors(local); + + expect(mockSetConnector).toHaveBeenCalledTimes(2); + expect(result.results).toEqual([ + { type: "gmail", action: "synced" }, + { type: "slack", action: "synced" }, + ]); + }); +}); + +const mockGetOAuthStatus = vi.mocked(api.getOAuthStatus); + +describe("runOAuthFlow", () => { + beforeEach(() => { + vi.resetAllMocks(); + }); + + it("returns ACTIVE when OAuth completes successfully", async () => { + const params: OAuthFlowParams = { + type: "gmail", + redirectUrl: "https://accounts.google.com/oauth", + connectionId: "conn_123", + }; + mockGetOAuthStatus.mockResolvedValue({ status: "ACTIVE" }); + + const result = await runOAuthFlow(params); + + expect(result).toEqual({ type: "gmail", status: "ACTIVE" }); + expect(mockGetOAuthStatus).toHaveBeenCalledWith("gmail", "conn_123"); + }); + + it("returns FAILED when OAuth fails", async () => { + const params: OAuthFlowParams = { + type: "gmail", + redirectUrl: "https://accounts.google.com/oauth", + connectionId: "conn_123", + }; + mockGetOAuthStatus.mockResolvedValue({ status: "FAILED" }); + + const result = await runOAuthFlow(params); + + expect(result).toEqual({ type: "gmail", status: "FAILED" }); + }); + + it("polls until status changes from PENDING", async () => { + const params: OAuthFlowParams = { + type: "gmail", + redirectUrl: "https://accounts.google.com/oauth", + connectionId: "conn_123", + }; + mockGetOAuthStatus + .mockResolvedValueOnce({ status: "PENDING" }) + .mockResolvedValueOnce({ status: "PENDING" }) + .mockResolvedValueOnce({ status: "ACTIVE" }); + + const result = await runOAuthFlow(params); + + expect(result).toEqual({ type: "gmail", status: "ACTIVE" }); + expect(mockGetOAuthStatus).toHaveBeenCalledTimes(3); + }); +}); diff --git a/tests/fixtures/invalid-connector/base44/.app.jsonc b/tests/fixtures/invalid-connector/base44/.app.jsonc new file mode 100644 index 00000000..d7852426 --- /dev/null +++ b/tests/fixtures/invalid-connector/base44/.app.jsonc @@ -0,0 +1,4 @@ +// Base44 App Configuration +{ + "id": "test-app-id" +} diff --git a/tests/fixtures/invalid-connector/config.jsonc b/tests/fixtures/invalid-connector/config.jsonc new file mode 100644 index 00000000..34efb6cf --- /dev/null +++ b/tests/fixtures/invalid-connector/config.jsonc @@ -0,0 +1,3 @@ +{ + "name": "Project with Invalid Connector" +} diff --git a/tests/fixtures/invalid-connector/connectors/invalid.jsonc b/tests/fixtures/invalid-connector/connectors/invalid.jsonc new file mode 100644 index 00000000..123665f8 --- /dev/null +++ b/tests/fixtures/invalid-connector/connectors/invalid.jsonc @@ -0,0 +1,5 @@ +// Invalid connector - unknown integration type +{ + "type": "invalid", + "scopes": [] +} diff --git a/tests/fixtures/with-connectors/base44/.app.jsonc b/tests/fixtures/with-connectors/base44/.app.jsonc new file mode 100644 index 00000000..d7852426 --- /dev/null +++ b/tests/fixtures/with-connectors/base44/.app.jsonc @@ -0,0 +1,4 @@ +// Base44 App Configuration +{ + "id": "test-app-id" +} diff --git a/tests/fixtures/with-connectors/config.jsonc b/tests/fixtures/with-connectors/config.jsonc new file mode 100644 index 00000000..c6c48d95 --- /dev/null +++ b/tests/fixtures/with-connectors/config.jsonc @@ -0,0 +1,3 @@ +{ + "name": "Project with Connectors" +} diff --git a/tests/fixtures/with-connectors/connectors/googlecalendar.jsonc b/tests/fixtures/with-connectors/connectors/googlecalendar.jsonc new file mode 100644 index 00000000..ee7a4c68 --- /dev/null +++ b/tests/fixtures/with-connectors/connectors/googlecalendar.jsonc @@ -0,0 +1,8 @@ +// Google Calendar connector +{ + "type": "googlecalendar", + "scopes": [ + "https://www.googleapis.com/auth/calendar.readonly", + "https://www.googleapis.com/auth/calendar.events" + ] +} diff --git a/tests/fixtures/with-connectors/connectors/notion.json b/tests/fixtures/with-connectors/connectors/notion.json new file mode 100644 index 00000000..ae65dfd1 --- /dev/null +++ b/tests/fixtures/with-connectors/connectors/notion.json @@ -0,0 +1,4 @@ +{ + "type": "notion", + "scopes": [] +} diff --git a/tests/fixtures/with-connectors/connectors/slack.jsonc b/tests/fixtures/with-connectors/connectors/slack.jsonc new file mode 100644 index 00000000..afb9c923 --- /dev/null +++ b/tests/fixtures/with-connectors/connectors/slack.jsonc @@ -0,0 +1,8 @@ +// Slack connector +{ + "type": "slack", + "scopes": [ + "chat:write", + "channels:read" + ] +}