use case scenario

[lamixer]

Hello! I have installed llama.cpp on a home ai server and I'd like to be able to talk to it instead of typing to it. I think I can configure my llama.cpp to behave like an OpenAI endpoint, so I think that I can talk to typewhisper-mac and it can talk to my llama. Right?

Then, how is the return trip made? Ideally, I'd have a choice of visual only or visual plus TTS (maybe via MacOS built-in mechanism) so I can 'converse' on my mac with my local llama machine.

Am I looking in the right place for this? Thank you!

[SeoFood]

Hi @lamixer, great question!

TypeWhisper can definitely help with this workflow - speech to text is the core feature, and it can process transcribed text through an LLM and even read back the response.

Here's how your setup could work:

Speech → Text → llama.cpp:

1. Install TypeWhisper and set up a local transcription engine (I'd recommend Parakeet for offline use)
2. Install the OpenAI Compatible plugin from the Plugin Marketplace (Settings → Integrations → Available)
3. Configure it to point to your llama.cpp server's OpenAI-compatible endpoint
4. Create a Custom Prompt (Settings → Prompts) and select your llama.cpp as the LLM provider
5. Use the Prompt Palette hotkey to dictate → transcribe → send to your llama → get the response inserted as text

The return trip (TTS):
TypeWhisper has a built-in Spoken Feedback mode (Settings → General → Accessibility) that reads back results using macOS text-to-speech. When enabled, it will automatically speak the LLM response after your prompt is processed - so you'd get a basic voice conversation loop out of the box.

So yes, you're looking in the right place! Hope that helps.

[lamixer]

Thank you Marco, I'll give it a try! As a side note, I have had a pretty good experience after setting up opwenweb-ui and using it to talk to (literally) the local LLM. I'm sure TypeWhisper will be even better and faster.