Voice Pipeline: Gemini Live audio not flowing through Twilio Media Streams

[madhavcodez]

Problem

When SecretAIRY places an outbound call via Twilio, the call connects and the Pipecat pipeline starts, but no audio is heard by the recipient. The Gemini Live LLM service connects successfully, but audio frames aren't flowing back through the Twilio WebSocket transport.

Root Cause Analysis

• Twilio fetches TwiML ✅
• WebSocket accepted ✅
• Pipecat pipeline links: Input → GeminiLive → Output ✅
• Gemini Live connects: "Connected to Gemini service" ✅
• Audio frames not reaching caller ❌

The connected and start events are consumed manually before handing the WebSocket to FastAPIWebsocketTransport. The transport may need to see these events, or there's an audio sample rate mismatch between Twilio (8kHz μ-law) and Gemini Live (16/24kHz PCM).

Attempted Fixes

• Fixed model name: gemini-2.5-flash-native-audio-preview → gemini-2.5-flash-native-audio-latest
• Disabled MachineDetection (was causing false-positive voicemail detection on trial accounts)
• Added <Say> TwiML opener before <Stream> for immediate audio feedback
• Audio sample rate alignment between TwilioFrameSerializer and Gemini Live
• Test if Pipecat's transport needs the raw start event to initialize audio routing

Steps to Reproduce

1. Start backend + Cloudflare tunnel
2. Create a campaign and click "Call Now"
3. Answer the phone, press key after trial message
4. Hear the <Say> opener but silence after stream connects

Logs

pipecat.services.google.gemini_live.llm:_connect - Connecting to Gemini service
pipecat.services.google.gemini_live.llm:_connection_task_handler - Connected to Gemini service
connection closed

Priority

P0 — Core feature blocker. Everything else works end-to-end.

Labels

bug, voice, pipecat, twilio