[Rhasspy2.5] make the app an audio client

[pajeronda]

I'm looking for a way to overcome a limitation of my current Rhasspy 2.5 setup connected to Home Assistant. I only use Rhasspy to send intents to Home Assistant, and in my current setup Home Assistant takes care of transforming intents that require audio with its TTS system. The audio generated with the Home Assistant TTS service is sent to a Snapcast server which plays it on my mobile devices. I operate Rhasspy with a smartphone and a tablet via the Rhasspy mobile app and everything works fine. The only downside is that I have to keep the Snapcast client app active in the background which occasionally fails and I can't hear anything. I would like to be able to use the Rhasspy mobile application to also play the audio that comes from the home assistant's TTS service or, alternatively, from the Snapcast server that I have installed on my local network, in order to avoid using the double application ( Rhasspy mobile + Snapcast client) on my mobile devices.

Can you make the app an audio client of Home Assistant or a Snapcast server?

Thank you

[Nailik]

Hi,

after generating your audio you could use the MQTT topic hermes/audioServer/<siteId>/playBytes/<requestId> (see https://github.com/Nailik/rhasspy_mobile/wiki/MQTT-Api) or the HTTP endpoint /api/play-wav (see https://github.com/Nailik/rhasspy_mobile/wiki/Http-Api) to play the audio via the app without the need of using snapcast.

This is an example subflow how to do this with http and nodered and a switch to send the audio to 3 different clients

[
    {
        "id": "7bcf4b8cd9b10281",
        "type": "http request",
        "z": "dc318e3192d7088b",
        "name": "",
        "method": "POST",
        "ret": "bin",
        "paytoqs": "ignore",
        "url": "http://client3:12101/api/play-wav",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [
            {
                "keyType": "other",
                "keyValue": "Content-Type",
                "valueType": "other",
                "valueValue": "audio/wav"
            }
        ],
        "x": 1310,
        "y": 540,
        "wires": [
            []
        ]
    },
    {
        "id": "9ca5c76ae65ef220",
        "type": "switch",
        "z": "dc318e3192d7088b",
        "name": "",
        "property": "siteId",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "client1",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "client2",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "client3",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 3,
        "x": 1090,
        "y": 480,
        "wires": [
            [
                "ea64e2537769ed63"
            ],
            [
                "252ecb4bc5f09c82"
            ],
            [
                "7bcf4b8cd9b10281"
            ]
        ]
    },
    {
        "id": "ea64e2537769ed63",
        "type": "http request",
        "z": "dc318e3192d7088b",
        "name": "",
        "method": "POST",
        "ret": "bin",
        "paytoqs": "ignore",
        "url": "http://client1:12101/api/play-wav",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [],
        "x": 1310,
        "y": 420,
        "wires": [
            []
        ]
    },
    {
        "id": "252ecb4bc5f09c82",
        "type": "http request",
        "z": "dc318e3192d7088b",
        "name": "",
        "method": "POST",
        "ret": "bin",
        "paytoqs": "ignore",
        "url": "http://client2:12101/api/play-wav",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [],
        "x": 1310,
        "y": 480,
        "wires": [
            []
        ]
    },
    {
        "id": "d3207bfa5a94d058",
        "type": "function",
        "z": "dc318e3192d7088b",
        "name": "build request",
        "func": "let text = msg.payload\nmsg.siteId = msg.payload.event._intent.siteId\nmsg.payload = { \"engine_id\": \"tts.piper\", \"message\": msg.text }\nreturn msg;",
        "outputs": 1,
        "timeout": "",
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 310,
        "y": 480,
        "wires": [
            [
                "d545c599538ba132"
            ]
        ]
    },
    {
        "id": "d545c599538ba132",
        "type": "http request",
        "z": "dc318e3192d7088b",
        "name": "get audio file",
        "method": "POST",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "homeassistant:8123/api/tts_get_url",
        "tls": "c02c32baa5028847",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "bearer",
        "senderr": false,
        "headers": [],
        "x": 510,
        "y": 480,
        "wires": [
            [
                "e19292acb369b8af"
            ]
        ]
    },
    {
        "id": "e19292acb369b8af",
        "type": "function",
        "z": "dc318e3192d7088b",
        "name": "set url for data",
        "func": "msg.url = 'homeassistant:8123' + msg.payload.path\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 720,
        "y": 480,
        "wires": [
            [
                "e6ec5e21e0f35e24"
            ]
        ]
    },
    {
        "id": "e6ec5e21e0f35e24",
        "type": "http request",
        "z": "dc318e3192d7088b",
        "name": "get data",
        "method": "GET",
        "ret": "bin",
        "paytoqs": "ignore",
        "url": "",
        "tls": "c02c32baa5028847",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "bearer",
        "senderr": false,
        "headers": [],
        "x": 920,
        "y": 480,
        "wires": [
            [
                "9ca5c76ae65ef220"
            ]
        ]
    }
]

[pajeronda]

Thank you very much, but I have a small problem: on Home Assistant, through a custom component I use Edge TTS, the TTS service of the Microsoft browser which works really well with the Italian language and offers the best "free" audio available. But this service returns an mp3 file that I pass to Snapcast.
This is why I asked you if it was possible to implement "media player" recognition in rhasspy mobile for Home Assistant or for Snapcast server.

[Nailik]

That's a great idea i'll add it to the backlog

[pajeronda]

thank you very much.
I look forward to trying out the new feature.