ElevenLabs Voice AI for Frappe

Integrates ElevenLabs Conversational AI with Frappe, enabling real-time two-way voice conversations with AI agents directly from your Frappe application.

Features

🎙️ Real-time Voice Calls - Talk to AI agents using WebRTC
🎯 Multiple Agents - Configure and manage multiple ElevenLabs agents
🖥️ Modern Vue 3 SPA - Beautiful, responsive voice interface
🔗 Frappe Integration - Launch calls directly from agent records
🔐 Secure - API keys stored encrypted, tokens are short-lived
📱 Mobile-Friendly - Works on desktop and mobile browsers

Tech Stack

Backend: Frappe (Python)
Frontend: Vue 3 + Vite + Tailwind CSS
Voice SDK: @elevenlabs/client (WebRTC)
Routing: Vue Router (clean URLs)

Installation

Prerequisites

Frappe framework installed
Node.js and Yarn
ElevenLabs account with API key

Step 1: Create a New Site (if needed)

cd /path/to/your/bench
bench new-site mysite.local
bench use mysite.local

Step 2: Get the App

# Clone from GitHub
bench get-app elevenlabs https://github.com/esafwan/frappe_elevenlabs.git --branch develop

Or if you have the app locally:

bench get-app elevenlabs /path/to/elevenlabs

Step 3: Install the App

bench --site mysite.local install-app elevenlabs

Step 4: Start Bench

bench start

Your app is now installed! 🎉

Configuration

Step 1: Configure ElevenLabs API Key

Access Frappe Desk:
```
http://localhost:8000/desk
```
Navigate to EL Provider Settings:
- Search for "EL Provider Settings" in the awesome bar (Ctrl+K)
- Or go directly: http://localhost:8000/app/el-provider-settings
Add your API Key:
- Get your API key from ElevenLabs Dashboard
- Paste it in the API Key field
- (Optional) Set Base URL if using a custom endpoint
- Click Save
Note: The API key is stored encrypted in the database for security.

Step 2: Create an Agent

Navigate to EL Agent List:
- Search for "EL Agent" in the awesome bar
- Or go directly: http://localhost:8000/app/el-agent
- Click New
Configure the Agent:
- Agent Name: FirstChapter (used in URLs, must be unique)
- ElevenLabs Agent ID: Get this from your ElevenLabs Agents page
  - Example: agent_2301k75k200zfvc9awgvrcdfk50g
- Click Save
Test Configuration:
- You should see a green status indicator if everything is configured correctly

Usage

Method 1: Quick Access Button (Recommended)

Open any EL Agent record in Desk
Click Actions → Open Voice Page
New tab opens with the voice interface
Click "Start Call" to begin talking to the AI agent

Method 2: Direct URL

Navigate directly to the voice interface using the URL format:

http://localhost:8000/voice/{agent-name}

Examples:

http://localhost:8000/voice/FirstChapter
http://localhost:8000/voice/CustomerSupport
http://localhost:8000/voice/MyAgent

Method 3: Portal Page

You can embed or link to the voice interface from:

Portal pages
Website pages
Custom web templates
Email links

Using the Voice Interface

Starting a Call

Load the page with an agent parameter
Click "Start Call" button
Allow microphone access when prompted
Start speaking - the AI will respond in real-time

During a Call

Mute/Unmute: Click the microphone button
View Timer: Shows call duration in MM:SS format
Status Indicator: Shows "Listening..." when active
End Call: Click the red phone button

Call States

State	Description	UI Indicator
Idle	Ready to start	Green "Available" badge
Connecting	Establishing connection	Spinner + "Connecting..."
Active	Call in progress	Red "In Call" badge, timer running
Error	Connection failed	Error message displayed

Architecture

URL Structure

/voice              → SPA entry point (shows warning if no agent)
/voice/AgentName    → Load specific agent and start interface

API Endpoints

Endpoint	Method	Purpose	Auth
`/api/method/elevenlabs.api.get_agent_signed_url`	GET	Load agent metadata	Guest allowed
`/api/method/elevenlabs.api.get_webrtc_token`	GET	Get signed WebSocket URL	Guest allowed

Data Flow

1. User opens /voice/FirstChapter
   ↓
2. Vue app loads agent metadata from Frappe DB
   ↓
3. User clicks "Start Call"
   ↓
4. Backend fetches signed URL from ElevenLabs
   ↓
5. Frontend connects to ElevenLabs via WebRTC
   ↓
6. Voice conversation starts

DocTypes

EL Provider Settings (Single)

Stores ElevenLabs API configuration:

API Key (Password, encrypted)
Base URL (optional, defaults to https://api.elevenlabs.io)

EL Agent

Represents a configured voice agent:

Agent Name (unique identifier)
ElevenLabs Agent ID (from ElevenLabs dashboard)

EL Voice Session (Future)

Planned for storing call history and analytics.

Development

Project Structure

elevenlabs/
├── frontend/                    # Vue 3 SPA
│   ├── src/
│   │   ├── pages/
│   │   │   └── Call.vue        # Main voice interface
│   │   ├── router.js           # Vue Router config
│   │   └── main.js             # App entry
│   ├── vite.config.js          # Vite configuration
│   └── package.json            # Frontend dependencies
│
└── elevenlabs/                  # Frappe app
    ├── api.py                   # Backend API endpoints
    ├── hooks.py                 # Frappe hooks
    ├── public/
    │   └── js/
    │       └── el_agent.js      # Custom button script
    └── www/
        ├── voice.html           # SPA entry (generated)
        └── voice.py             # Page handler

Building the Frontend

# Development mode (hot reload)
cd /path/to/bench/apps/elevenlabs
yarn dev

# Production build
yarn build

# After building, restart bench
cd /path/to/bench
bench restart

Adding Features

Modify Vue components in frontend/src/
Build frontend: yarn build
Clear cache: bench clear-cache
Restart: bench restart

Troubleshooting

"ElevenLabs API Key not configured"

Solution: Configure the API key in EL Provider Settings:

Desk → EL Provider Settings → Add API Key → Save

"Failed to load agent"

Causes:

Agent doesn't exist in database
Wrong agent name in URL

Solution:

Check agent exists: Desk → EL Agent → {agent-name}
Verify agent name matches URL parameter exactly

"No signed_url received from ElevenLabs"

Causes:

Invalid API key
Invalid agent ID
Network/firewall issues

Solution:

Verify API key in EL Provider Settings
Check agent ID is correct in EL Agent record
Check error logs: Desk → Error Log

Microphone permission denied

Solution:

Allow microphone access when prompted
Check browser settings for microphone permissions
HTTPS required in production (localhost OK for dev)

Page shows 404

Solution:

# Rebuild and clear cache
cd /path/to/bench
bench build --app elevenlabs
bench clear-cache
bench restart

Call connects but AI can't hear me

Check:

Microphone is not muted (check icon)
Browser has microphone permission
Correct microphone selected in browser settings
No other app is using the microphone

Browser Compatibility

Browser	Status	Notes
Chrome/Edge (Chromium)	✅ Fully supported	Recommended
Firefox	✅ Fully supported
Safari (macOS/iOS)	✅ Supported	Test on Apple devices
Internet Explorer 11	❌ Not supported	No WebRTC support

Security

API Key Storage

Stored encrypted in database using Frappe's Password field
Never exposed to frontend
Retrieved server-side only

WebRTC Tokens

Short-lived (~1 minute)
Single-use signed URLs
Fetched fresh for each call

Public Access

Voice interface allows guest access
Consider adding authentication for production
Rate limiting recommended

Production Deployment

Pre-deployment Checklist

Configure production API key
Test on production domain with HTTPS
Verify all agents are configured
Test on multiple browsers
Test on mobile devices
Set up error monitoring
Configure rate limiting (if needed)
Enable SSL/HTTPS (required for microphone access)

HTTPS Requirement

Development: Works with localhost (no HTTPS needed)

Production: HTTPS required for:

Microphone access (getUserMedia API)
WebRTC connections
Service workers (future)

Contributing

Pre-commit Hooks

This app uses pre-commit for code formatting and linting:

cd apps/elevenlabs
pre-commit install

Tools configured:

ruff - Python linting
pyupgrade - Python syntax upgrades
eslint - JavaScript linting
prettier - Code formatting

Code Style

Python: Follow Frappe conventions
JavaScript/Vue: Use Composition API, async/await
CSS: Tailwind utility classes
Commits: Descriptive, conventional commits

Links

ElevenLabs Dashboard: https://elevenlabs.io/app
API Documentation: https://elevenlabs.io/docs/api-reference
Frappe Framework: https://frappeframework.com/docs
GitHub Repository: https://github.com/esafwan/frappe_elevenlabs

Support

For issues or questions:

Check the troubleshooting section above
Review error logs: Desk → Error Log
Check browser console (F12) for frontend errors
Open an issue on GitHub

License

AGPL-3.0

Quick Reference

# Installation
bench get-app elevenlabs https://github.com/esafwan/frappe_elevenlabs.git
bench --site mysite install-app elevenlabs

# Development
cd apps/elevenlabs
yarn dev              # Frontend dev server
yarn build            # Production build
bench restart         # Restart backend

# Access
http://localhost:8000/desk                    # Frappe Desk
http://localhost:8000/app/el-provider-settings # Configure API
http://localhost:8000/app/el-agent            # Manage agents
http://localhost:8000/voice/AgentName         # Voice interface

Built with ❤️ using Frappe Framework and ElevenLabs Conversational AI

Name		Name	Last commit message	Last commit date
Latest commit History 19 Commits
docs		docs
elevenlabs		elevenlabs
frontend		frontend
.editorconfig		.editorconfig
.eslintrc		.eslintrc
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
AGENTS.md		AGENTS.md
README.md		README.md
license.txt		license.txt
package.json		package.json
pyproject.toml		pyproject.toml
yarn.lock		yarn.lock

License

tridz-dev/frappe_ElevenLabs

Folders and files

Latest commit

History

Repository files navigation

ElevenLabs Voice AI for Frappe

Features

Tech Stack

Installation

Prerequisites

Step 1: Create a New Site (if needed)

Step 2: Get the App

Step 3: Install the App

Step 4: Start Bench

Configuration

Step 1: Configure ElevenLabs API Key

Step 2: Create an Agent

Usage

Method 1: Quick Access Button (Recommended)

Method 2: Direct URL

Method 3: Portal Page

Using the Voice Interface

Starting a Call

During a Call

Call States

Architecture

URL Structure

API Endpoints

Data Flow

DocTypes

EL Provider Settings (Single)

EL Agent

EL Voice Session (Future)

Development

Project Structure

Building the Frontend

Adding Features

Troubleshooting

"ElevenLabs API Key not configured"

"Failed to load agent"

"No signed_url received from ElevenLabs"

Microphone permission denied

Page shows 404

Call connects but AI can't hear me

Browser Compatibility

Security

API Key Storage

WebRTC Tokens

Public Access

Production Deployment

Pre-deployment Checklist

HTTPS Requirement

Contributing

Pre-commit Hooks

Code Style

Links

Support

License

Quick Reference

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages