TypeError: 'NoneType' object is not callable

[Jaid844]

I am attempting to create a server using the example_fast_api folder, but I keep receiving this error in Runpod.

Initializing openai engine Traceback (most recent call last): File "/workspace/RealtimeTTS/example_fast_api/server.py", line 428, in <module> engines["openai"] = OpenAIEngine() TypeError: 'NoneType' object is not callable

I thought the issue might be from the openAI class, then I removed this class from the server.py file, and then I got this error from the system engine.

Traceback (most recent call last): File "/workspace/RealtimeTTS/example_fast_api/server.py", line 416, in <module> engines["system"] = SystemEngine() TypeError: 'NoneType' object is not callable

Perhaps I am doing something incorrectly. here have added my openai_api_key as well. Any help will be appreciated

[Jaid844]

I have tried the README.md example as well,

from RealtimeTTS import OpenAIEngine , TextToAudioStream

engine = OpenAIEngine() # replace with your TTS engine
stream = TextToAudioStream(engine)
stream.feed("Hello world! How are you today?")
stream.play_async()

Replaced with openAI TTS still the error presist!

C:\Users\91639\.conda\envs\transcr\lib\site-packages\pydub\utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning) Traceback (most recent call last): File "C:\Users\91639\Desktop\Transcription_sockets\rough.py", line 3, in <module> engine = OpenAIEngine() # replace with your TTS engine TypeError: 'NoneType' object is not callable

[KoljaB]

For openai pls do pip install realtimetts[openai], fir system engine pip install realtimetts[system]. Then it should work.

[Jaid844]

Hi @KoljaB, Do you have any blog/resource recommendations, perhaps a YouTube video on scaling RealtimeTTS horizontally? I just want to try with 2 GPUs and several users .

[KoljaB]

There are no resources online I'm aware of. I suggest use CUDA_VISIBLE_DEVICES to assign RealtimeTTS instances to each GPU, serve with fastapi, perform load balancing depending on the engine.

[Jaid844]

Hi @KoljaB, I’d like to get your opinion. Do you think it's more practical to capture and play the audio chunks as they come in, rather than collecting all of them first and then playing them? GPT seems to fumble with question :)

[KoljaB]

Don't collect, that adds too much latency. Subscribe to on_audio_chunk callback. Put all those incoming chunks in a queue. Process that queue from another thread and send every chunk to the client as fast as possible. The client should play every received chunk immediately.

[Jaid844]

Hi @KoljaB,
I built a WebSocket using the websocket-client library — I figured it would help me get more familiar with the library and possibly reduce latency. But I’ve run into an odd issue: the audio does play, but two different versions come through — one sounds fine, and the other sounds totally demonic 😂

Here the server side code

async def data_handler(websocket):
    print(f"{bcolors.OKGREEN}Data client connected{bcolors.ENDC}")
    data_connections.add(websocket)
    try:
        while True:
            message = await websocket.recv()
            if play_text_to_speech_semaphore.acquire(blocking=False):
                threading.Thread(
                    target=play_text_to_speech, args=(stream, message, audio_queue), daemon=True
                ).start()
                async for audio_chunk in audio_chunk_generator(audio_queue, None):
                    if audio_chunk:
                        await websocket.send(audio_chunk)
            else:
                await websocket.send(
                    "Service unavailable, currently processing another request. Please try again shortly."
                )
                continue
    except websockets.exceptions.ConnectionClosed as e:
        print(f"{bcolors.WARNING}Data client disconnected: {e}{bcolors.ENDC}")
    finally:
        data_connections.remove(websocket)
        await websocket.close()
        
        
        async def main_async():
    try:
        print("Initializing TTS Engines")

        for engine_name in SUPPORTED_ENGINES:
            if "openai" == engine_name:
                print("Initializing openai engine")
                engines["openai"] = OpenAIEngine()

        for _engine in engines.keys():
            print(f"Retrieving voices for TTS Engine {_engine}")
            try:
                voices[_engine] = engines[_engine].get_voices()
            except Exception as e:
                voices[_engine] = []
                logging.error(f"Error retrieving voices for {_engine}: {str(e)}")

        _set_engine(START_ENGINE)
        data_server = await websockets.serve(data_handler, "localhost", 8500)
        print(f"{bcolors.OKGREEN}Voice websocket server started on {bcolors.OKBLUE}ws://localhost:{8500}{bcolors.ENDC}")
        await data_server.wait_closed()
    except OSError:
        print(
            f"{bcolors.FAIL}Error: Could not start server on specified ports. It’s possible another instance of the server is already running, or the ports are being used by another application.{bcolors.ENDC}")


def main():
    try:
        asyncio.run(main_async())
    except KeyboardInterrupt:
        # Capture any final KeyboardInterrupt to prevent it from showing up in logs
        print(f"{bcolors.WARNING}Server interrupted by user.{bcolors.ENDC}")
        exit(0)  # Sucessfully terminates the program


if __name__ == "__main__":
    main()

Here is the client side code

import websocket
import pyaudio
import threading
import time


FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 22050
CHUNK = 1024


p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                output=True)


WS_URL = "ws://127.0.0.1:8500"
TEXT_TO_SEND = "Hello, how are you doing>"

def on_open(ws):
    print("[WebSocket] Connection opened")
    
    
    def send_text():
        time.sleep(1)
        print("[WebSocket] Sending text:", TEXT_TO_SEND)
        ws.send(TEXT_TO_SEND)
    
    threading.Thread(target=send_text).start()

def on_message(ws, message):
    # If message is binary (audio), play it
    if isinstance(message, bytes):
        stream.write(message)
    else:
        print("[WebSocket] Received text message:", message)

def on_error(ws, error):
    print("[WebSocket] Error:", error)

def on_close(ws, close_status_code, close_msg):
    print("[WebSocket] Closed", close_status_code, close_msg)
    stream.stop_stream()
    stream.close()
    p.terminate()

if __name__ == "__main__":
    ws_app = websocket.WebSocketApp(WS_URL,
                                    on_open=on_open,
                                    on_message=on_message,
                                    on_error=on_error,
                                    on_close=on_close)

    ws_app.run_forever()

[KoljaB]

Can I see the full server code? Hard to reproduce...

[Jaid844]

Apologies for the late response.

I'm currently working on integrating RealtimeSTT with RealtimeTTS, but I'm running into an issue that says:

no close frame received or sent

Below is the server-side WebSocket code for RealtimeTTS.

I've gone through multiple Stack Overflow posts, but I haven't been able to find a solution so far.

import argparse
import asyncio
import os

from websocket import ABNF

parser = argparse.ArgumentParser(description="Run the TTS FastAPI server.")
parser.add_argument("-p", "--port", type=int, default=int(os.environ.get("TTS_FASTAPI_PORT", 8000)),
                    help="Port to run the FastAPI server on (default: 8000 or TTS_FASTAPI_PORT env var).")
parser.add_argument('-D', '--debug', action='store_true', help='Enable debug logging for detailed server operations')

args = parser.parse_args()

PORT = args.port
DEBUG_LOGGING = args.debug

if __name__ == "__main__":
    import logging

    if DEBUG_LOGGING:
        logging.basicConfig(level=logging.DEBUG)
    else:
        logging.basicConfig(level=logging.WARNING)

if __name__ == "__main__":
    print(f"Starting server on port {PORT}")

from RealtimeTTS import (
    TextToAudioStream,

    OpenAIEngine,

)

from fastapi.responses import StreamingResponse, HTMLResponse, FileResponse
from fastapi.middleware.cors import CORSMiddleware
from fastapi import FastAPI, Query, Request, HTTPException, WebSocket, WebSocketDisconnect
from fastapi.staticfiles import StaticFiles

from queue import Queue
import threading
import logging
import uvicorn
import wave
import io

SUPPORTED_ENGINES = [
    "openai",
    "system",
]

# change start engine:
START_ENGINE = SUPPORTED_ENGINES[0]

BROWSER_IDENTIFIERS = [
    "mozilla",
    "chrome",
    "safari",
    "firefox",
    "edge",
    "opera",
    "msie",
    "trident",
]

origins = [
    "http://localhost",
    f"http://localhost:{PORT}",
    "http://127.0.0.1",
    f"http://127.0.0.1:{PORT}",
    "https://localhost",
    f"https://localhost:{PORT}",
    "https://127.0.0.1",
    f"https://127.0.0.1:{PORT}",
]

audio_queue = Queue()
play_text_to_speech_semaphore = threading.Semaphore(1)
engines = {}
voices = {}
current_engine = None
stream = None
current_speaking = {}
speaking_lock = threading.Lock()
tts_lock = threading.Lock()
gen_lock = threading.Lock()

app = FastAPI()
app.mount("/static", StaticFiles(directory="static"), name="static")
app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Define a CSP that allows 'self' for script sources for firefox
csp = {
    "default-src": "'self'",
    "script-src": "'self'",
    "style-src": "'self' 'unsafe-inline'",
    "img-src": "'self' data:",
    "font-src": "'self' data:",
    "media-src": "'self' blob:",
}
csp_string = "; ".join(f"{key} {value}" for key, value in csp.items())


@app.middleware("http")
async def add_security_headers(request: Request, call_next):
    response = await call_next(request)
    response.headers["Content-Security-Policy"] = csp_string
    return response


@app.get("/favicon.ico")
async def favicon():
    return FileResponse("static/favicon.ico")


def _set_engine(engine_name):
    global current_engine, stream
    print(f"Current engine{current_engine}")

    if current_engine is None:
        current_engine = engines[engine_name]
        stream = TextToAudioStream(current_engine, muted=False)
    else:
        current_engine = engines[engine_name]
        stream.load_engine(current_engine)

    if voices[engine_name]:
        engines[engine_name].set_voice(voices[engine_name][0].name)


@app.get("/set_engine")
def set_engine(request: Request, engine_name: str = Query(...)):
    if engine_name not in engines:
        return {"error": "Engine not supported"}

    try:
        _set_engine(engine_name)
        return {"message": f"Switched to {engine_name} engine"}
    except Exception as e:
        logging.error(f"Error switching engine: {str(e)}")
        return {"error": "Failed to switch engine"}


def run_asyncio_coroutine(coroutine, *args):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_until_complete(coroutine(*args))


def play_text_to_speech(stream, text, audio_queue):
    set_speaking(text, True)

    def on_audio_chunk(chunk):
        logging.debug("Received chunk")
        audio_queue.put(chunk)

    try:
        stream.feed(text)
        logging.debug(f"Playing audio for text: {text}")
        print(f'Synthesizing: "{text}"')
        stream.play(on_audio_chunk=on_audio_chunk, muted=False)
        audio_queue.put(None)
    finally:
        set_speaking(text, False)
        play_text_to_speech_semaphore.release()


def is_browser_request(request):
    user_agent = request.headers.get("user-agent", "").lower()
    is_browser = any(browser_id in user_agent for browser_id in BROWSER_IDENTIFIERS)
    return is_browser


def create_wave_header_for_engine(engine):
    _, _, sample_rate = engine.get_stream_info()

    num_channels = 1
    sample_width = 2
    frame_rate = sample_rate

    wav_header = io.BytesIO()
    with wave.open(wav_header, "wb") as wav_file:
        wav_file.setnchannels(num_channels)
        wav_file.setsampwidth(sample_width)
        wav_file.setframerate(frame_rate)

    wav_header.seek(0)
    wave_header_bytes = wav_header.read()
    wav_header.close()

    # Create a new BytesIO with the correct MIME type for Firefox
    final_wave_header = io.BytesIO()
    final_wave_header.write(wave_header_bytes)
    final_wave_header.seek(0)

    return final_wave_header.getvalue()


async def audio_chunk_generator(audio_queue, send_wave_headers):
    with gen_lock:
        first_chunk = False
        try:
            while True:
                chunk = audio_queue.get()
                if chunk is None:
                    logging.debug("Terminating stream")
                    break
                if not first_chunk:
                    if (
                            send_wave_headers
                            and not current_engine.engine_name == "elevenlabs"
                    ):
                        logging.debug("Sending wave header")
                        yield create_wave_header_for_engine(current_engine)
                    first_chunk = True
                logging.debug("Sending chunk")
                yield chunk
        except Exception as e:
            logging.error(f"Error during streaming: {str(e)}")


def is_currently_speaking(text):
    with speaking_lock:
        return current_speaking.get(text, False)


def set_speaking(text, status):
    with speaking_lock:
        current_speaking[text] = status


@app.get("/tts")
def tts(request: Request, text: str = Query(...)):
    browser_request = is_browser_request(request)
    audio_queue = Queue()

    if play_text_to_speech_semaphore.acquire(blocking=False):
        threading.Thread(
            target=play_text_to_speech, args=(stream, text, audio_queue), daemon=True
        ).start()
    else:
        raise HTTPException(
            status_code=503,
            detail="Service unavailable, currently processing another request. Please try again shortly.",
            headers={"Retry-After": "10"},
        )

    return StreamingResponse(
        audio_chunk_generator(audio_queue, browser_request),
        media_type="audio/wav"
        if current_engine.engine_name != "elevenlabs"
        else "audio/mpeg",
    )


import websockets

data_connections = set()


class bcolors:
    HEADER = '\033[95m'  # Magenta
    OKBLUE = '\033[94m'  # Blue
    OKCYAN = '\033[96m'  # Cyan
    OKGREEN = '\033[92m'  # Green
    WARNING = '\033[93m'  # Yellow
    FAIL = '\033[91m'  # Red
    ENDC = '\033[0m'  # Reset to default
    BOLD = '\033[1m'
    UNDERLINE = '\033[4m'


async def data_handler(websocket):
    print(f"{bcolors.OKGREEN}Data client connected{bcolors.ENDC}")
    data_connections.add(websocket)
    try:
        while True:
            message = await websocket.recv()
            if play_text_to_speech_semaphore.acquire(blocking=False):
                threading.Thread(
                    target=play_text_to_speech, args=(stream, message, audio_queue), daemon=True
                ).start()
                async for audio_chunk in audio_chunk_generator(audio_queue, None):
                    if audio_chunk:
                        await websocket.send(audio_chunk)
            else:
                await websocket.send(
                    "Service unavailable, currently processing another request. Please try again shortly."
                )
                continue
    except websockets.exceptions.ConnectionClosed as e:
        print(f"{bcolors.WARNING}Data client disconnected: {e}{bcolors.ENDC}")
    finally:
        data_connections.remove(websocket)
        await websocket.close()



@app.get("/tts-text")
def tts_text(request: Request, text: str = Query(...)):
    if "favicon.ico" in request.url.path:
        print("favicon requested")
        return FileResponse("static/favicon.ico")

    print(f"/tts_text route synthesizing text: {text}")

    browser_request = is_browser_request(request)

    if play_text_to_speech_semaphore.acquire(blocking=False):
        threading.Thread(
            target=play_text_to_speech, args=(stream, text), daemon=True
        ).start()
    else:
        logging.debug("Can't play audio, another instance is already running")

    return StreamingResponse(
        audio_chunk_generator(browser_request), media_type="audio/wav"
    )


@app.get("/engines")
def get_engines():
    return list(engines.keys())


@app.get("/voices")
def get_voices():
    voices_list = []
    for voice in voices[current_engine.engine_name]:
        voices_list.append(voice.name)
    return voices_list


@app.get("/setvoice")
def set_voice(request: Request, voice_name: str = Query(...)):
    print(f"Getting request: {voice_name}")
    if not current_engine:
        print("No engine is currently selected")
        return {"error": "No engine is currently selected"}

    try:
        print(f"Setting voice to {voice_name}")
        current_engine.set_voice(voice_name)
        return {"message": f"Voice set to {voice_name} successfully"}
    except Exception as e:
        print(f"Error setting voice: {str(e)}")
        logging.error(f"Error setting voice: {str(e)}")
        return {"error": "Failed to set voice"}


@app.get("/")
def root_page():
    engines_options = "".join(
        [
            f'<option value="{engine}">{engine.title()}</option>'
            for engine in engines.keys()
        ]
    )
    content = f"""
    <!DOCTYPE html>
    <html>
        <head>
            <title>Text-To-Speech</title>
            <style>
                body {{
                    font-family: Arial, sans-serif;
                    background-color: #f0f0f0;
                    margin: 0;
                    padding: 0;
                }}
                h2 {{
                    color: #333;
                    text-align: center;
                }}
                #container {{
                    width: 80%;
                    margin: 50px auto;
                    background-color: #fff;
                    border-radius: 10px;
                    padding: 20px;
                    box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
                }}
                label {{
                    font-weight: bold;
                }}
                select, textarea {{
                    width: 100%;
                    padding: 10px;
                    margin: 10px 0;
                    border: 1px solid #ccc;
                    border-radius: 5px;
                    box-sizing: border-box;
                    font-size: 16px;
                }}
                button {{
                    display: block;
                    width: 100%;
                    padding: 15px;
                    background-color: #007bff;
                    border: none;
                    border-radius: 5px;
                    color: #fff;
                    font-size: 16px;
                    cursor: pointer;
                    transition: background-color 0.3s;
                }}
                button:hover {{
                    background-color: #0056b3;
                }}
                audio {{
                    width: 80%;
                    margin: 10px auto;
                    display: block;
                }}
            </style>
        </head>
        <body>
            <div id="container">
                <h2>Text to Speech</h2>
                <label for="engine">Select Engine:</label>
                <select id="engine">
                    {engines_options}
                </select>
                <label for="voice">Select Voice:</label>
                <select id="voice">
                    <!-- Options will be dynamically populated by JavaScript -->
                </select>
                <textarea id="text" rows="4" cols="50" placeholder="Enter text here..."></textarea>
                <button id="speakButton">Speak</button>
                <audio id="audio" controls></audio> <!-- Hidden audio player -->
            </div>
            <script src="/static/tts.js"></script>
        </body>
    </html>
    """
    return HTMLResponse(content=content)


async def main_async():
    try:
        print("Initializing TTS Engines")

        for engine_name in SUPPORTED_ENGINES:
            if "openai" == engine_name:
                print("Initializing openai engine")
                engines["openai"] = OpenAIEngine()

        for _engine in engines.keys():
            print(f"Retrieving voices for TTS Engine {_engine}")
            try:
                voices[_engine] = engines[_engine].get_voices()
            except Exception as e:
                voices[_engine] = []
                logging.error(f"Error retrieving voices for {_engine}: {str(e)}")

        _set_engine(START_ENGINE)
        data_server = await websockets.serve(data_handler, "localhost", 8500)
        print(f"{bcolors.OKGREEN}Voice websocket server started on {bcolors.OKBLUE}ws://localhost:{8500}{bcolors.ENDC}")
        await data_server.wait_closed()
    except OSError:
        print(
            f"{bcolors.FAIL}Error: Could not start server on specified ports. It’s possible another instance of the server is already running, or the ports are being used by another application.{bcolors.ENDC}")


def main():
    try:
        asyncio.run(main_async())
    except KeyboardInterrupt:
        # Capture any final KeyboardInterrupt to prevent it from showing up in logs
        print(f"{bcolors.WARNING}Server interrupted by user.{bcolors.ENDC}")
        exit(0)  # Sucessfully terminates the program


if __name__ == "__main__":
    main()

here is the client side code

 import json
import sys
import threading
import time
from urllib.parse import urlparse
import socket
import pyaudio
import resampy
import numpy as np
from websocket import WebSocketApp
import pyaudio._portaudio as pa

DEFAULT_VOICE_URL = "ws://127.0.0.1:8500"
import logging


class bcolors:
    HEADER = '\033[95m'  # Magenta
    OKBLUE = '\033[94m'  # Blue
    OKCYAN = '\033[96m'  # Cyan
    OKGREEN = '\033[92m'  # Green
    WARNING = '\033[93m'  # Yellow
    FAIL = '\033[91m'  # Red
    ENDC = '\033[0m'  # Reset to default
    BOLD = '\033[1m'
    UNDERLINE = '\033[4m'


class Audio_configuration:

    def __init__(self,
                 format: int = pyaudio.paInt16,
                 channels: int = 1,
                 rate: int = 16000,
                 output_device_index=None,
                 frames_per_buffer: int = pa.paFramesPerBufferUnspecified,
                 playout_chunk_size: int = -1,
                 ):
        self.format = format
        self.channels = channels
        self.rate = rate
        self.output_device_index = output_device_index
        self.frames_per_buffer = frames_per_buffer
        self.playout_chunk_size = playout_chunk_size


config1 = Audio_configuration(
    65536,
    1,
    22050,
    None,
    frames_per_buffer=pa.paFramesPerBufferUnspecified,
    playout_chunk_size=-1,
)


class AudioStream:
    """
    Handles audio stream operations
    - opening, starting, stopping, and closing
    """

    def __init__(self, config: Audio_configuration):
        """
        Args:
            config (AudioConfiguration): Object containing audio settings.
        """
        self.config = config
        self.stream = None
        self.pyaudio_instance = pyaudio.PyAudio()
        self.actual_sample_rate = 0
        self.mpv_process = None


    def get_supported_sample_rates(self, device_index):
        """
        Test which standard sample rates are supported by the specified device.

        Args:
            device_index (int): The index of the audio device to test

        Returns:
            list: List of supported sample rates
        """
        standard_rates = [8000, 9600, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000]
        supported_rates = []

        device_info = self.pyaudio_instance.get_device_info_by_index(device_index)
        max_channels = device_info.get('maxOutputChannels')

        # Test each standard sample rate
        for rate in standard_rates:
            try:
                if self.pyaudio_instance.is_format_supported(
                        rate,
                        output_device=device_index,
                        output_channels=max_channels,
                        output_format=self.config.format,
                ):
                    supported_rates.append(rate)
            except:
                continue
        return supported_rates

    def _get_best_sample_rate(self, device_index, desired_rate):
        """
        Determines the best available sample rate for the device.

        Args:
            device_index: Index of the audio device
            desired_rate: Preferred sample rate

        Returns:
            int: Best available sample rate
        """
        try:
            # First determine the actual device index to use
            actual_device_index = (device_index if device_index is not None
                                   else self.pyaudio_instance.get_default_output_device_info()['index'])

            # Now use the actual_device_index for getting device info and supported rates
            device_info = self.pyaudio_instance.get_device_info_by_index(actual_device_index)
            supported_rates = self.get_supported_sample_rates(actual_device_index)

            # Check if desired rate is supported
            if desired_rate in supported_rates:
                return desired_rate

            # Find the highest supported rate that's lower than desired_rate
            lower_rates = [r for r in supported_rates if r <= desired_rate]
            if lower_rates:
                return max(lower_rates)

            # If no lower rates, get the lowest higher rate
            higher_rates = [r for r in supported_rates if r > desired_rate]
            if higher_rates:
                return min(higher_rates)

            # If nothing else works, return device's default rate
            return int(device_info.get('defaultSampleRate', 44100))

        except Exception as e:
            logging.warning(f"Error determining sample rate: {e}")
            return 44100

    def open_stream(self):
        """Opens an audio stream."""

        # check for mpeg format
        pyChannels = self.config.channels
        desired_rate = self.config.rate
        pyOutput_device_index = self.config.output_device_index

        best_rate = self._get_best_sample_rate(pyOutput_device_index, desired_rate)
        self.actual_sample_rate = best_rate
        if self.config.format == pyaudio.paCustomFormat:
            pyFormat = self.pyaudio_instance.get_format_from_width(2)
            print(f"pyFormat {pyFormat}")
            logging.info(
                "Opening stream for mpeg audio chunks, "
                f"pyFormat: {pyFormat}, pyChannels: {pyChannels}, "
                f"pySampleRate: {best_rate}"
            )
        else:
            pyFormat = self.config.format
            logging.debug(
                "Opening stream for wave audio chunks, "
                f"pyFormat: {pyFormat}, pyChannels: {pyChannels}, "
                f"pySampleRate: {best_rate}"
            )
        try:
            self.stream = self.pyaudio_instance.open(
                format=pyFormat,
                channels=pyChannels,
                rate=best_rate,
                output_device_index=pyOutput_device_index,
                frames_per_buffer=self.config.frames_per_buffer,
                output=True,
            )
        except Exception as e:
            print(
                "Error opening stream with parameters:"
                f" format={pyFormat}, channels={pyChannels}, rate={best_rate}, output_device_index={pyOutput_device_index}"
                f"Error message: {e}")

    def start_stream(self):
        """Starts the audio stream."""
        if self.stream and not self.stream.is_active():
            self.stream.start_stream()

    def stop_stream(self):
        """Stops the audio stream."""
        if self.stream and self.stream.is_active():
            self.stream.stop_stream()

    def close_stream(self):
        """Closes the audio stream."""
        if self.stream:
            self.stop_stream()
            self.stream.close()
            self.stream = None


class AITextVoice:
    def __init__(self, config: Audio_configuration = config1):
        self.data_ws_thread = None
        print("dodd")
        self.data_ai_url = DEFAULT_VOICE_URL
        self.connection_established = threading.Event()
        self.audio_stream = AudioStream(config)

    def start(self):
        if not self.connect():  # tries to open a websocket connection to server
            print("Failed to connect to the server.", file=sys.stderr)

    def _on_data_message(self, ws, chunk):
        try:
            print("data_message")
            audio_data = np.frombuffer(chunk, dtype=np.int16)
            audio_data = audio_data.astype(np.float32) / 32768.0
            resampled_data = resampy.resample(audio_data, self.audio_stream.config.rate,
                                              self.audio_stream.actual_sample_rate)
            chunk = (resampled_data * 32768.0).astype(np.int16).tobytes()

            sub_chunk_size = 512
            sample_width = self.audio_stream.pyaudio_instance.get_sample_size(self.audio_stream.config.format)
            channels = self.audio_stream.config.channels

            for i in range(0, len(chunk), sub_chunk_size):
                sub_chunk = chunk[i: i + sub_chunk_size]
                frames_in_sub_chunk = len(sub_chunk) // (sample_width * channels)
                start_time = time.time()
                timeout = 0.1
                while self.audio_stream.stream.get_write_available() < frames_in_sub_chunk:
                    if time.time() - start_time > timeout:
                        break
                    time.sleep(0.001)
                self.audio_stream.stream.write(sub_chunk)
        except Exception as e:
            print(f"Error in audio playback: {e}")

    def is_server_running(self):
        parsed_url = urlparse(self.data_ai_url)
        host = parsed_url.hostname
        port = parsed_url.port or 80
        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
            return s.connect_ex((host, port)) == 0

    def ensure_server_running(self):
        if not self.is_server_running():
            for _ in range(20):  # Wait up to 20 seconds
                if self.is_server_running():
                    time.sleep(2)  # Give the server a moment to fully initialize
                    return True
                time.sleep(1)
            print("Failed to start Voice server.", file=sys.stderr)
            return False

    def on_error(self, ws, error):
        print(f"WebSocket error: {error}")

    def on_close(self, ws, close_status_code, close_msg):
        if ws == self.data_ai_ws:
            logging.info(f"Voice WebSocket connection closed: {close_status_code} - {close_msg}")

    def on_data_open(self, ws):
        print("Voice WebSocket connection opened.")
        self.connection_established.set()

    def connect(self):
        # if not self.ensure_server_running():
        #    print("Cannot connect Voice server. Exiting.")
        #    return False

        try:
            self.data_ai_ws = WebSocketApp(self.data_ai_url,
                                           on_message=self._on_data_message,
                                           on_error=self.on_error,  # calls this method whenever a error happens
                                           on_close=self.on_close,
                                           on_open=self.on_data_open)

            self.data_ws_thread = threading.Thread(target=self.data_ai_ws.run_forever)
            self.data_ws_thread.daemon = False
            self.data_ws_thread.start()
            print(f"{bcolors.OKBLUE}Voice websocket connections established successfully.{bcolors.ENDC}")

            if not self.connection_established.wait(timeout=10):
                print("Timeout while connecting to the server.")
                return False
            return True
        except Exception as e:
            print(f"Error while connecting to the server: {e}")
            return False

    def shutdown(self):
        if self.data_ai_ws:
            self.data_ai_ws.close()
        if self.data_ws_thread:
            self.data_ws_thread.join()

        self.audio_stream.close_stream()

        print(f"{bcolors.WARNING} Voice websocket connection has been disconnected{bcolors.ENDC}")

[Jaid844]

Hi @KoljaB, I think I figured out the issue — it was related to self.recording_start. I moved the WebSocket class initialization above the main method in audio_cli_client.py, and that seemed to do the trick.

[KoljaB]

Ok, great, thanks for the info.

[Jaid844]

Hi @KoljaB I am trying to create class orpheus instead of running in LM studio ,I am using orpheus model from orpheus-voice as inference but i am getting issue from ffmeg like

"/usr/local/lib/python3.11/dist-packages/pydub/audio_segment.py", line 773, in from_file raise CouldntDecodeError( pydub.exceptions.CouldntDecodeError: Decoding failed. ffmpeg returned error code: 1 Output from ffmpeg/avlib:.

Any pointers please 😅