Excalidraw Copilot Drawing App

An Electron-based macOS application that combines GitHub Copilot SDK with Excalidraw to create an AI-powered drawing assistant. Describe what you want to draw in natural language, and the AI will generate Excalidraw diagrams for you.

Features

• 🤖 AI-Powered Drawing: Use natural language to describe diagrams and drawings
• ✏️ Excalidraw Integration: Full-featured drawing canvas with Excalidraw
• 💬 Conversational Interface: Interactive chat with streaming responses
• 💾 Session Persistence: Automatic saving of conversations and drawings
• 🔐 GitHub Authentication: Secure authentication via GitHub CLI or API key
• 📦 SQLite Storage: Efficient metadata storage with better-sqlite3

Prerequisites

• Node.js 18+ and npm
• GitHub CLI (gh) or GitHub API key with Copilot access
• macOS (other platforms may work but are untested)

Installation

1. Navigate to the project directory:

cd excalidraw-copilot-app

2. Install dependencies (already done if you followed setup):

npm install

3. Authenticate with GitHub:

gh auth login

Usage

Development Mode

Run the app in development mode with hot-reload:

npm run dev

This will:

1. Start the Vite dev server on http://localhost:5173
2. Launch Electron with the app
3. Open DevTools automatically

Building for Production

Build the app for distribution:

npm run electron:build

This creates a distributable app in the dist folder.

How It Works

Architecture

The app is split into two main processes:

1. Main Process (Electron):

   • Manages Copilot SDK client
   • Handles authentication
   • Stores session data and drawings
   • Provides IPC interface

2. Renderer Process (React):

   • Chat interface for user interaction
   • Excalidraw canvas for drawing
   • JSON parser for AI responses
   • Real-time updates via IPC

Drawing Commands

The AI responds with JSON in markdown code blocks:

{
  "elements": [
    {
      "type": "rectangle",
      "x": 100,
      "y": 100,
      "width": 200,
      "height": 150,
      "text": "Server",
      "strokeColor": "#000000",
      "backgroundColor": "#ffffff"
    },
    {
      "type": "arrow",
      "startX": 300,
      "startY": 175,
      "endX": 500,
      "endY": 175
    }
  ]
}

Supported element types:

• rectangle, ellipse, diamond: Shapes with optional text
• arrow, line: Connectors between elements
• text: Standalone text labels

Storage

• Copilot Sessions: Managed by SDK (stored in XDG_STATE_HOME)
• Excalidraw Scenes: JSON files in ~/Library/Application Support/excalidraw-copilot-app/drawings/
• Session Metadata: SQLite database in user data directory

Example Prompts

Try these prompts with the AI:

• "Draw a simple flowchart with start, process, and end boxes"
• "Create a system architecture diagram with 3 servers connected to a load balancer"
• "Make a mind map about project planning"
• "Draw a sequence diagram showing a user login flow"
• "Create an org chart with 5 team members"

Project Structure

excalidraw-copilot-app/
├── electron/                 # Main process code
│   ├── main.ts              # Entry point
│   ├── auth.ts              # Authentication manager
│   ├── copilot.ts           # Copilot SDK wrapper
│   ├── storage.ts           # SQLite + file storage
│   └── preload.ts           # IPC bridge
├── src/                     # Renderer process code
│   ├── components/          # React components
│   │   ├── ChatPanel.tsx    # Chat interface
│   │   └── ChatPanel.css
│   ├── utils/               # Utilities
│   │   └── parser.ts        # JSON parser & converter
│   ├── types/               # TypeScript definitions
│   ├── App.tsx              # Main app component
│   ├── App.css
│   └── main.tsx             # React entry point
├── index.html               # HTML entry
├── vite.config.ts           # Vite configuration
├── tsconfig.json            # TypeScript config
└── package.json

Configuration

Vite Configuration

The vite.config.ts includes:

• React plugin for JSX/TSX support
• Electron plugin for main process compilation
• Electron renderer plugin for preload script

TypeScript Configuration

Two tsconfig files:

• tsconfig.json: Main TypeScript config for src and electron
• tsconfig.node.json: Config for Vite and build tools

Troubleshooting

Authentication Issues

If you see authentication errors:

1. Verify GitHub CLI is installed: gh --version
2. Check authentication status: gh auth status
3. Re-authenticate: gh auth login

Build Errors

If you encounter build errors:

1. Clear node_modules: rm -rf node_modules && npm install
2. Clear Vite cache: rm -rf node_modules/.vite
3. Check Node version: node --version (should be 18+)

Excalidraw Not Loading

If the canvas doesn't appear:

1. Check browser console for errors
2. Verify Excalidraw package is installed: npm ls @excalidraw/excalidraw
3. Try restarting the dev server

Development

Running Tests

npm test

Linting

npm run lint

Type Checking

npx tsc --noEmit

Security Considerations

1. Authentication: Uses GitHub CLI credentials stored securely by gh
2. IPC: Context isolation enabled in preload script
3. Node Integration: Disabled in renderer for security
4. Storage: Session data stored in user's protected app data directory

Limitations

• Currently optimized for macOS (Windows/Linux untested)
• Requires active GitHub Copilot subscription
• Drawing complexity limited by AI model capabilities
• No real-time collaboration features

Future Enhancements

• Windows and Linux support
• Export drawings as PNG/SVG
• Undo/redo with full scene history
• Custom element templates
• Multi-session tab interface
• Improved AI prompt engineering
• Error recovery and retry logic
• Performance optimizations for large drawings

License

ISC

Contributing

Contributions welcome! Please open issues for bugs or feature requests.

Acknowledgments

• GitHub Copilot SDK
• Excalidraw
• Electron
• Vite
• React