(help) TTS with LMStudio

[oivio]

What I am trying to do is run simple tts chat with loaded qwen2-0.5b-instruct at this moment I try to look and understand at test examples simple_llm_test.py

import os
import openai  # pip install openai
from RealtimeTTS import TextToAudioStream, SystemEngine

openai.api_key = os.environ.get("OPENAI_API_KEY")


def write(prompt: str):
    for chunk in openai.ChatCompletion.create(
        model="qwen2-0.5b-instruct",
        messages=[{"role": "user", "content": prompt}],
        stream=True,
    ):
        if (text_chunk := chunk["choices"][0]["delta"].get("content")) is not None:
            yield text_chunk


text_stream = write("A three-sentence relaxing speech.")

print("Starting to play")

TextToAudioStream(SystemEngine()).feed(text_stream).play()

Maybe because its 2am and my brain cant think. Maybe I do have to create LMStudioEngine?
Any guidance will help.

[KoljaB]

Please try this code:

from openai import OpenAI
from RealtimeTTS import TextToAudioStream, SystemEngine

engine = SystemEngine()
stream = TextToAudioStream(engine)
client = OpenAI(base_url="http://localhost:1234/v1")

def write(prompt: str):
    for chunk in client.chat.completions.create(
        model="qwen2-0.5b-instruct",
        messages=[{"role": "user", "content": prompt}],
        stream=True,
    ):
        text_chunk = chunk.choices[0].delta.content
        if text_chunk:
            yield text_chunk

text_stream = write("A three-sentence relaxing speech.")

stream.feed(text_stream)
stream.play()

[oivio]

Thank you @KoljaB but, sadly it leave me openai error:

Traceback (most recent call last):
  File "e:\GitHub\RealtimeTTS\tests\simple_llm_test_LMStudio.py", line 6, in <module>
    client = OpenAI(base_url="http://localhost:1234/v1")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\GitHub\RealtimeTTS\env_realtimetts\Lib\site-packages\openai\_client.py", line 114, in __init__
    raise OpenAIError(
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

looks like issue with api_key, I will try create maybe lmstudio_engine the goal is just to have that kind of speech2speech conversation :)

Still you have here real solid amazing work <3

[KoljaB]

Pls try client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

[oivio]

thank you @KoljaB <3

I did have to define client var so final working code is:

from openai import OpenAI
from RealtimeTTS import TextToAudioStream, SystemEngine

engine = SystemEngine()
stream = TextToAudioStream(engine)

client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")
    
def write(prompt: str):
    for chunk in client.chat.completions.create(
        model="qwen2-0.5b-instruct",
        messages=[{"role": "user", "content": prompt}],
        stream=True,
    ):
        text_chunk = chunk.choices[0].delta.content
        if text_chunk:
            yield text_chunk

text_stream = write("A three-sentence relaxing speech.")

stream.feed(text_stream)
stream.play()