An open-source voice-powered AI assistant for Ray-Ban Meta smart glasses. 50+ built-in tools, multi-LLM support (cloud + on-device), personas with simultaneous wake words, live translation, MCP tool servers, and more — all controlled hands-free by voice.
Note: The Meta Wearables SDK is currently in developer preview. App Store distribution is not yet supported — each user must build the app from source with their own Meta developer credentials.
- Build and install on your iPhone (see Building from Source)
- Add an AI model in Settings → AI Models (Anthropic, OpenAI, Gemini, or a local model)
- Pair your Ray-Ban Meta glasses via the Meta AI app
- Say "Hey OpenGlasses" and ask anything
Each persona has its own wake word, AI model, and personality. All listen simultaneously.
| Say | What Happens |
|---|---|
| "Hey Claude" | Routes to Claude Sonnet with your professional prompt |
| "Hey Jarvis" | Routes to a local on-device model with a concise style |
| "Hey Computer" | Routes to GPT-4o with a technical personality |
Configure: Settings → Personas → Add. Pick a wake word, assign a model and prompt preset.
Run AI models entirely on your iPhone — no internet, no cloud, no API keys.
- Settings → AI Models → Add Model → pick "Local (On-Device)"
- Download & Manage Models → download from HuggingFace
- Select your downloaded model and tap Add
Recommended models:
| Model | Size | Best For |
|---|---|---|
| Qwen 2.5 3B | 1.8 GB | Conversation + tool calling |
| Gemma 2 2B | 1.5 GB | General purpose |
| SmolVLM2 2.2B | 1.5 GB | Vision (can see photos) |
| Qwen 2.5 0.5B | 0.4 GB | Ultra-light, basic |
Models are stored persistently and work fully offline after download. Toggle Offline Mode in Settings → Tools to disable internet-dependent tools.
All voice-activated. Say what you need naturally — the AI picks the right tool.
| Category | Tools |
|---|---|
| Information | Web Search (Perplexity + DuckDuckGo), News, Weather, Date/Time, Dictionary, Currency |
| Productivity | Calendar, Reminders, Alarms, Timers, Pomodoro, Notes, Contextual Notes (GPS+time tagged), Clipboard |
| Communication | Phone Calls, iMessage, WhatsApp, Telegram, Email, Contact Lookup |
| Navigation | Directions (Apple/Google Maps), Nearby Places, Save Locations, Geofencing Alerts |
| Media | Music Control (play/pause/skip + search by song/artist), Shazam Song ID, Open Apps |
| Smart Home | HomeKit (lights, switches, fans, thermostats, locks, scenes), Home Assistant (REST API), Siri Shortcuts |
| Vision | QR/Barcode Scanner, Face Recognition |
| Memory | Object Memory ("where are my keys?"), Social Context (per-person facts), User Memory, Voice-Taught Skills |
| AI Features | Live Translation, Memory Rewind (ambient audio recall), Ambient Captions, Meeting Summaries, Conversation Summaries |
| Fitness | Workout Tracking, Exercise Logging, HealthKit, Pose Analysis, Step Goals |
| Device | Flashlight, Brightness, Device Info, Step Count |
| Safety | Emergency Info (local numbers + GPS), Daily Briefing, Navigation Assistance (accessibility preset) |
| Integration | OpenClaw Gateway (50+ skills), MCP Servers (universal tool protocol), Custom Tools |
Teach the AI new behaviors at runtime — no code needed.
| Say | What Happens |
|---|---|
| "Learn that when I say expense this, create a note tagged EXPENSE" | Skill saved, auto-applies forever |
| "Learn that when I say goodnight, turn off all lights" | Triggers HomeKit/HA on the phrase |
| "List skills" | Shows all taught skills |
| "Forget expense this" | Removes the skill |
Remember where you put things. Uses GPS to calculate distance.
| Say | What Happens |
|---|---|
| "Remember my car is in lot B level 3" | Saves with GPS + timestamp |
| "Where are my keys?" | "Your keys were on the kitchen counter, 2 hours ago. That's very close to where you are now." |
| "Where did I park?" | Retrieves car location with distance |
Continuous real-time translation of spoken foreign language.
| Say | What Happens |
|---|---|
| "Start translating Spanish to English" | Begins continuous translation |
| "Stop translating" | Ends session, reports count |
| "Switch to Japanese to English" | Changes languages on the fly |
Supports 25+ languages including Spanish, French, German, Japanese, Chinese, Korean, Arabic, and more.
Build dossiers about people you meet.
| Say | What Happens |
|---|---|
| "Remember Sarah works at Google and likes hiking" | Fact saved |
| "What do I know about Sarah?" | "About Sarah: works at Google, likes hiking. First noted 3 days ago." |
Works alongside face recognition — when the AI recognizes someone, it can recall your notes about them.
Interrupt the AI mid-sentence by saying any wake word. It stops immediately and starts listening to your new question.
Switch AI personality without reconfiguring. Built-in presets:
| Preset | Style |
|---|---|
| Default | Balanced, 2-4 sentences, conversational |
| Concise | 1-2 sentences max, no filler |
| Technical | Precise, jargon-appropriate, data-dense |
| Creative | Playful, witty, expressive |
| Navigation Aid | Spatial awareness, obstacle detection, sign reading |
Create your own in Settings → System Prompt.
Define new tools without writing code. Map to Siri Shortcuts or URL schemes.
Settings → Transparency → Custom Tools → Add:
- Shortcut tool: triggers a Siri Shortcut by name
- URL tool: opens a URL with parameter substitution
Example: a "log_water" tool that runs your "Log Water" shortcut when the AI decides you need it.
Connect to any MCP-compatible tool server directly from your phone.
Settings → Transparency → MCP Servers → Add:
- Enter server URL + auth headers
- Tap "Discover Tools" — all tools auto-appear
- The AI can call them alongside native tools
Popular MCP servers: Home Assistant, Notion, GitHub, Slack, Todoist, and hundreds more.
Direct REST API control of your HA instance — works alongside or instead of HomeKit.
Settings → Services → Home Assistant:
- HA URL: e.g.
http://192.168.1.100:8123 - Token: Long-Lived Access Token (HA → Profile → Security)
Voice commands: "Turn on the living room lights", "Set thermostat to 72", "Run the goodnight automation", "List all sensors"
See exactly what data the AI receives and what network calls are made.
| Setting | What It Shows |
|---|---|
| Tools | All 50+ tools with enable/disable toggles |
| Prompt Inspector | Full system prompt, injected context, token estimate |
| Network Activity | All HTTP requests categorized by Meta/AI/App/Other |
| Offline Mode | One toggle disables all internet-requiring tools |
- Voice-Activated Photo Capture — "take a picture" or "what's this?"
- QR/Barcode Scanner — "scan this code" (Vision framework, works offline)
- Live Camera Preview — real-time view of glasses POV
- Video Recording — MP4 with configurable bitrate
- RTMP Broadcasting — live stream to YouTube, Twitch, Kick
- WebRTC Browser Streaming — shareable URL for peer-to-peer viewing
- Privacy Filter — auto-blurs bystander faces
24 ElevenLabs voices (10 female, 14 male) with iOS fallback:
- Female: Rachel, Sarah, Matilda, Emily, Charlotte, Alice, Lily, Dorothy, Serena, Nicole
- Male: Brian, Adam, Daniel, George, Chris, Charlie, James, Dave, Drew, Callum, Bill, Fin, Liam, Thomas
Emotion-Aware TTS adjusts tone automatically — warmer for good news, calmer for instructions, concerned for warnings.
| Mode | How It Works |
|---|---|
| Voice Mode | Wake word → transcription → any LLM → TTS (most flexible) |
| Gemini Live | Real-time audio/video streaming with Google Gemini |
| OpenAI Realtime | Real-time audio/video streaming with OpenAI |
- iOS 17+ (built targeting iOS 26)
- Xcode 15+
- Physical iPhone (Bluetooth, camera, microphone required)
- Ray-Ban Meta smart glasses (paired via Meta AI app)
- At least one LLM: API key (Anthropic, OpenAI, Gemini, etc.) OR a downloaded local model
git clone https://github.com/straff2002/OpenGlasses.git
cd OpenGlasses- Go to wearables.developer.meta.com
- Create an account, organization, and app
- Note your Meta App ID and Client Token
- In Meta dashboard → iOS settings, enter your Apple Team ID, Bundle ID, and Universal Link URL
Update OpenGlasses/Info.plist:
<key>MWDAT</key>
<dict>
<key>AppLinkURLScheme</key>
<string>https://YOUR-DOMAIN/YOUR-PATH</string>
<key>MetaAppID</key>
<string>YOUR_META_APP_ID</string>
<key>ClientToken</key>
<string>AR|YOUR_META_APP_ID|YOUR_CLIENT_TOKEN_HASH</string>
<key>TeamID</key>
<string>$(DEVELOPMENT_TEAM)</string>
</dict>Host an apple-app-site-association file at https://YOUR-DOMAIN/.well-known/apple-app-site-association:
{
"applinks": {
"details": [{
"appID": "YOUR_TEAM_ID.YOUR_BUNDLE_ID",
"paths": ["/YOUR-PATH/*"]
}]
}
}On iPhone: Meta AI app → Settings → About → tap version number 5 times → toggle Developer Mode on.
open OpenGlasses.xcodeprojSelect your iPhone, set your Team in Signing, and run (⌘R).
All settings are in-app — no source code editing needed.
| Service | Purpose | Where to Get |
|---|---|---|
| Anthropic | Claude LLM | console.anthropic.com |
| OpenAI | GPT + Realtime | platform.openai.com |
| Google Gemini | Gemini Live | aistudio.google.com |
| Groq | Fast inference | console.groq.com |
| ElevenLabs | Natural TTS | elevenlabs.io |
| Perplexity | Web search | perplexity.ai/settings/api |
| Service | Settings |
|---|---|
| ElevenLabs | API key + voice selection (24 voices) |
| Perplexity | API key (DuckDuckGo fallback if not set) |
| Live Streaming | Platform + RTMP URL + stream key |
| OpenClaw | Enable + connection mode + host/port + token |
| Home Assistant | URL + Long-Lived Access Token |
| Issue | Solution |
|---|---|
| Wake word not detecting | Tap mic button to restart; check Bluetooth audio routing |
| No audio through glasses | Verify Bluetooth connection in iOS Settings |
| Glasses not connecting | Tap "Connect to Glasses"; enable Developer Mode in Meta AI app |
| HomeKit not finding devices | HomeKit initializes on first tool call — say "list smart home devices" and wait 10s |
| Local model crashes | Use a smaller model (0.5B or 2B); the 3B model may OOM on 6GB devices |
| Model download stuck | Keep app in foreground; downloads continue if briefly backgrounded |
| "Untrusted Developer" | Settings → General → VPN & Device Management → Verify (requires internet) |
| Package | Purpose |
|---|---|
| meta-wearables-dat-ios | Glasses connection + camera |
| HaishinKit | RTMP broadcasting |
| mlx-swift-lm | On-device LLM inference |
Contributions welcome! This is fully open-source. Fork, improve, submit PRs.
Key areas for contribution:
- New native tools
- Local model optimization
- Translation quality improvements
- Additional MCP server integrations
- UI/UX improvements
Business Source License 1.1 — free for non-commercial use. Commercial use requires a separate license from Skunk0 / Skunkworks NZ. Converts to Apache 2.0 on March 24, 2030. See LICENSE file for details.
Built by Skunk0 at Skunkworks NZ
Powered by Anthropic Claude, Meta Wearables SDK, Apple MLX, ElevenLabs, HaishinKit
Note: Independent open-source project, not affiliated with Meta or Anthropic.