Pull Request / Add MQTT and persistent connection for real time

[mrwiwi]

Hi, still haven't processed how to use github, so if you could have a look at my code here :

It's a WIP of your client interfaced with a mqtt broker (here on homeassistant), i'm still learning python and programming in general, but it's working :)

Hard time on parsing data, can't use the type arg anymore (websocket is listened permanently, so we can have real time update of our devices), we need or more adaptable function.

Please someone review it, we're not far from a clean client i think.

Auto reconnect is implemented.

Output sample :
`S C:\Users\lipit> & C:/bin/Python38-32/python.exe d:/Drive/Domotique/pydom/pydom.py
##################################
Websocket is closed or inexistant, trying....
"Attempting websocket connection..."
Tydom Websocket is Connected ! <websockets.client.WebSocketClientProtocol object at 0x0435C760>
Attempting MQTT connection...
##################################
MQTT is connected ! =)
Subscribing to : homeassistant/+/tydom/#
##################################
Requesting 1st data...
MQTT SUBSCRIBED

1234568029 Baie
1234568089 Panoramique
1234568149 Cosina
1234568209 Salle De Bain
1234568269 Chambre Parents
1234568329 Chambre Quentin
1234568389 Chambre 3
1234591388 Alarme
Cannot parse response
string indices must be integers

Cannot parse response
list indices must be integers or slices, not str

MQTT incoming: homeassistant/sensor/tydom/last_update b'2019-12-11 00:32:54.442518'

MQTT incoming: homeassistant/sensor/tydom/last_update b'2019-12-11 00:32:54.557796'

MQTT incoming: homeassistant/requests/tydom/update b'please'

1234568029 Baie
1234568089 Panoramique
1234568149 Cosina
1234568209 Salle De Bain
1234568269 Chambre Parents
1234568329 Chambre Quentin
1234568389 Chambre 3
1234591388 Alarme
Cannot parse response`

Here it is !

https://pastebin.com/xy9R2RGk

[cth35]

Hi great job ;)
I will have a look to it and will merge it to the trunk.
Thanks a lot for your contribution.

[mrwiwi]

Hi great job ;)
I will have a look to it and will merge it to the trunk.
Thanks a lot for your contribution.

Thanks a lot !!

I've updated everything, now with handling of error, single parsing every kind of message, still figuring out some put commands but hey, it's stable as hell :)

https://pastebin.com/xssNyj5s

#!/usr/bin/env python
import asyncio
import websockets
import http.client
from requests.auth import HTTPDigestAuth
import sys
import logging
from http.client import HTTPResponse
from io import BytesIO
import urllib3
import json
import os
import base64
import time
from http.server import BaseHTTPRequestHandler
import ssl
from datetime import datetime
from gmqtt import Client as MQTTClient


# Globals
####################################### MQTT
tydom_topic = "homeassistant/+/tydom/#"

cover_config_topic = "homeassistant/cover/tydom/{id}/config"
cover_config = "homeassistant/cover/tydom/{id}/config"
cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"


alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
alarm_command_topic = "homeassistant/alarm_control_panel/tydom/{id}/set"
alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

refresh_topic = "homeassistant/tydom/please_update"

mac = "" #sys.argv[2]
login = mac
password = "" #sys.argv[3]
host = "mediation.tydom.com" #"192.168.0.20" # Local ip address or mediation.tydom.com for remote connexion

mqtt_host = ''
mqtt_user = ''
mqtt_pass = ''
mqtt_port = 8883
mqtt_ssl = True
#INIT Servers
hassio = None
tydom = None

# Set Host, ssl context and prefix for remote or local connection
if host == "mediation.tydom.com":
    remote_mode = True
    ssl_context = None
    cmd_prefix = "\x02"
else:
    remote_mode = False
    ssl_context = ssl._create_unverified_context()
    cmd_prefix = ""

deviceAlarmKeywords = ['alarmMode','alarmState','alarmSOS','zone1State','zone2State','zone3State','zone4State','zone5State','zone6State','zone7State','zone8State','gsmLevel','inactiveProduct','zone1State','liveCheckRunning','networkDefect','unitAutoProtect','unitBatteryDefect','unackedEvent','alarmTechnical','systAutoProtect','sysBatteryDefect','zsystSupervisionDefect','systOpenIssue','systTechnicalDefect','videoLinkDefect']
# Device dict for parsing
device_dict = dict()


#MQTT
STOP = asyncio.Event()
def on_connect(client, flags, rc, properties):
    print("##################################")
    
    print("Subscribing to : ", tydom_topic)
    # client.subscribe('homeassistant/#', qos=0)
    client.subscribe(tydom_topic, qos=0)
    
async def on_message(client, topic, payload, qos, properties):
    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
    # print('MQTT incoming : ', topic, payload)
    if (topic == "homeassistant/requests/tydom/update") or (payload == "please"):
        await get_data(tydom)
    if ('set_position' in str(topic)):
        print('MQTT set_position incoming : ', topic, payload)
        get_id = (topic.split("/"))[3] #extract id from mqtt
        await put_devices_data(tydom, str(get_id), 'position', str(payload))
    else:
        return 0

async def on_disconnect(client, packet, exc=None):
    print('MQTT Disconnected')
    print("##################################")
    await main_task()

def on_subscribe(client, mid, qos):
    print("MQTT is connected and suscribed ! =)")



def ask_exit(*args):
    STOP.set()

async def mqttconnection(broker_host, user, password):
    
    global hassio
    if (hassio == None):
        print('Attempting MQTT connection...')
        client = MQTTClient("client-id")

        client.on_connect = on_connect
        client.on_message = on_message
        client.on_disconnect = on_disconnect
        client.on_subscribe = on_subscribe

        client.set_auth_credentials(user, password)
        await client.connect(broker_host, port=mqtt_port, ssl=mqtt_ssl)
        hassio = client
        
    # client.publish('TEST/TIME', str(time.time()), qos=1)

    # await STOP.wait()
    # await client.disconnect()


#######" END MQTT"

class Cover:
    def __init__(self, id, name, current_position=None, set_position=None, attributes=None):
        
        self.id = id
        self.name = name
        self.current_position = current_position
        self.set_position = set_position
        self.attributes = attributes

    def id(self):
        return self.id

    def name(self):
        return self.name

    def current_position(self):
        return self.current_position

    def set_position(self):
        return self.set_position

    def attributes(self):
        return self.attributes

    # cover_config_topic = "homeassistant/cover/tydom/{id}/config"
    # cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
    # cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
    # cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"

    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Volet'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_topic = cover_config_topic.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['set_position_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['position_topic'] = cover_position_topic.format(id=self.id)
        self.config['payload_open'] = 100
        self.config['payload_close'] = 0
        self.config['retain'] = 'true'
        self.config['device'] = self.device

        # print(self.config)
        hassio.publish(self.config_topic, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.position_topic = cover_position_topic.format(id=self.id, current_position=self.current_position)
        hassio.publish(self.position_topic, self.current_position, qos=0, retain=True)

        # self.attributes_topic = cover_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)

class Alarm:
    def __init__(self, id, name, current_state=None, attributes=None):
        self.id = id
        self.name = name
        self.current_state = current_state
        self.attributes = attributes

    # def id(self):
    #     return id

    # def name(self):
    #     return name

    # def current_state(self):
    #     return current_state

    # def attributes(self):
    #     return attributes
    
    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Tyxal'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_alarm = alarm_config.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        self.config['device'] = self.device
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = alarm_command_topic.format(id=self.id)
        self.config['state_topic'] = alarm_state_topic.format(id=self.id)


        # print(self.config)
        hassio.publish(self.config_alarm, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.state_topic = alarm_state_topic.format(id=self.id, state=self.current_state)
        hassio.publish(self.state_topic, self.current_state, qos=0, retain=True)

        # self.attributes_topic = alarm_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)
    

    # alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
    # alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
    # alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
    # alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
    # alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

class BytesIOSocket:
    def __init__(self, content):
        self.handle = BytesIO(content)

    def makefile(self, mode):
        return self.handle

class HTTPRequest(BaseHTTPRequestHandler):
    def __init__(self, request_text):
        #self.rfile = StringIO(request_text)
        self.raw_requestline = request_text
        self.error_code = self.error_message = None
        self.parse_request()

    def send_error(self, code, message):
        self.error_code = code
        self.error_message = message

def response_from_bytes(data):
    sock = BytesIOSocket(data)
    response = HTTPResponse(sock)
    response.begin()
    return urllib3.HTTPResponse.from_httplib(response)

def put_response_from_bytes(data):
    request = HTTPRequest(data)
    return request

# Get pretty name for a device id
def get_name_from_id(id):
    name = ""
    if len(device_dict) != 0:
        name = device_dict[id]
    return(name)

# Basic response parsing. Typically GET responses
async def parse_response(incoming):
    data = incoming
    msg_type = None
    first = str(data[:20])
    
    # Detect type of incoming data
    if (data != ''):
        if ("id" in first):
            print('Incoming message type : data detected')
            msg_type = 'msg_data'
        elif ("date" in first):
            print('Incoming message type : config detected')
            msg_type = 'msg_config'
        elif ("doctype" in first):
            print('Incoming message type : html detected (probable pong)')
            msg_type = 'msg_html'
            print(data)
        else:
            print('Incoming message type : no type detected')
            print(first)

        
        if not (msg_type == None):
            try:
                parsed = json.loads(data)
                # print(parsed)
                if (msg_type == 'msg_config'):
                    for i in parsed["endpoints"]:
                        # Get list of shutter
                        if i["last_usage"] == 'shutter':
                            # print('{} {}'.format(i["id_endpoint"],i["name"]))
                            device_dict[i["id_endpoint"]] = i["name"]
                                                
                            # TODO get other device type
                        if i["last_usage"] == 'alarm':
                            # print('{} {}'.format(i["id_endpoint"], i["name"]))
                            device_dict[i["id_endpoint"]] = "Tyxal Alarm"
                    print('Configuration updated')
                elif (msg_type == 'msg_data'):
                    for i in parsed:
                        attr = {}
                        if i["endpoints"][0]["error"] == 0:
                            for elem in i["endpoints"][0]["data"]:
                                # Get full name of this id
                                endpoint_id = i["endpoints"][0]["id"]
                                # Element name
                                elementName = elem["name"]
                                # Element value
                                elementValue = elem["value"]
                                
                                # Get last known position (for shutter)
                                if elementName == 'position':
                                    name_of_id = get_name_from_id(endpoint_id)
                                    if len(name_of_id) != 0:
                                        print_id = name_of_id
                                    else:
                                        print_id = endpoint_id
                                    # print('{} : {}'.format(print_id, elementValue))
                                    new_cover = "cover_tydom_"+str(endpoint_id)
                                    print("Cover created / updated : "+new_cover)
                                    new_cover = Cover(id=endpoint_id,name=print_id, current_position=elementValue, attributes=i)
                                    new_cover.update()

                                # Get last known position (for alarm)
                                if elementName in deviceAlarmKeywords:
                                    alarm_data = '{} : {}'.format(elementName, elementValue)
                                    # print(alarm_data)
                                    # alarmMode  : ON or ZONE or OFF
                                    # alarmState : ON = Triggered
                                    # alarmSOS   : true = SOS triggered
                                    state = None
                                    sos = False
                                    
                                    if alarm_data == "alarmMode : ON":
                                        state = "armed_away"
                                    elif alarm_data == "alarmMode : ZONE":
                                        state = "armed_home"
                                    elif alarm_data == "alarmMode : OFF":
                                        state = "disarmed"
                                    elif alarm_data == "alarmState : ON":
                                        state = "triggered"
                                    elif alarm_data == "alarmSOS : true":
                                        sos = True
                                    else:
                                        attr[elementName] = [elementValue]
                                    #     attr[alarm_data]
                                        # print(attr)
                                    #device_dict[i["id_endpoint"]] = i["name"]
                                    if (sos == True):
                                        print("SOS !")
                                    if not (state == None):
                                        # print(state)
                                        alarm = "alarm_tydom_"+str(endpoint_id)
                                        print("Alarm created / updated : "+alarm)
                                        alarm = Alarm(id=endpoint_id,name="Tyxal Alarm", current_state=state, attributes=attr)
                                        alarm.update()
                elif (msg_type == 'msg_html'):
                    print("pong")
                else:
                    # Default json dump
                    print()
                    print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))
            except Exception as e:
                print('Cannot parse response !')
                # print('Response :')
                # print(data)
                if (e != 'Expecting value: line 1 column 1 (char 0)'):
                    print(e)


# PUT response DIRTY parsing
def parse_put_response(bytes_str):
    # TODO : Find a cooler way to parse nicely the PUT HTTP response
    resp = bytes_str[len(cmd_prefix):].decode("utf-8")
    fields = resp.split("\r\n")
    fields = fields[6:]  # ignore the PUT / HTTP/1.1
    end_parsing = False
    i = 0
    output = str()
    while not end_parsing:
        field = fields[i]
        if len(field) == 0 or field == '0':
            end_parsing = True
        else:
            output += field
            i = i + 2
    parsed = json.loads(output)
    return json.dumps(parsed)
    # print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))

# Generate 16 bytes random key for Sec-WebSocket-Keyand convert it to base64
def generate_random_key():
    return base64.b64encode(os.urandom(16))

# Build the headers of Digest Authentication
def build_digest_headers(nonce):
    digestAuth = HTTPDigestAuth(login, password)
    chal = dict()
    chal["nonce"] = nonce[2].split('=', 1)[1].split('"')[1]
    chal["realm"] = "ServiceMedia" if remote_mode is True else "protected area"
    chal["qop"] = "auth"
    digestAuth._thread_local.chal = chal
    digestAuth._thread_local.last_nonce = nonce
    digestAuth._thread_local.nonce_count = 1
    return digestAuth.build_digest_header('GET', "https://{}:443/mediation/client?mac={}&appli=1".format(host, mac))

# Send Generic GET message
async def send_message(websocket, msg):
    str = cmd_prefix + "GET " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv() #disable if handler

# Send Generic POST message
async def send_post_message(websocket, msg):
    str = cmd_prefix + "POST " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv()


###############################################################
# Commands                                                    #
###############################################################

# Get some information on Tydom
async def get_info(websocket):
    msg_type = '/info'
    parse_response(await send_message(websocket, msg_type), msg_type)

# Refresh (all)
async def post_refresh(websocket):
    print("Refresh....")
    msg_type = '/refresh/all'
    await send_post_message(websocket, msg_type)

# Get the moments (programs)
async def get_moments(websocket):
    msg_type = '/moments/file'
    await send_message(websocket, msg_type)

# Get the scenarios
async def get_scenarios(websocket):
    msg_type = '/scenarios/file'
    await send_message(websocket, msg_type)

# Get a ping (pong should be returned)
async def get_ping(websocket):
    msg_type = 'ping'
    await send_message(websocket, msg_type)

# Get all devices metadata
async def get_devices_meta(websocket):
    msg_type = '/devices/meta'
    parse_response(await send_message(websocket, msg_type), msg_type)

# Get all devices data
async def get_devices_data(websocket):
    msg_type = '/devices/data'
    await send_message(websocket, msg_type)

# List the device to get the endpoint id
async def get_configs_file(websocket):
    msg_type = '/configs/file'
    await send_message(websocket, msg_type)


async def get_data(websocket):
    await get_configs_file(websocket)
    await asyncio.sleep(2)
    await get_devices_data(websocket)

# Give order (name + value) to endpoint
async def put_devices_data(websocket, endpoint_id, name, value):
    # For shutter, value is the percentage of closing
    body="[{\"name\":\"" + name + "\",\"value\":\""+ value + "\"}]"
    # endpoint_id is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "PUT /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: ".format(str(endpoint_id),str(endpoint_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)
    # name = await websocket.recv()
    # try:
    #     parse_response(name)
    # except:
    #     parse_put_response(name)

# Run scenario
async def put_scenarios(websocket, scenario_id):
    body=""
    # scenario_id is the id of scenario got from the get_scenarios command
    str_request = cmd_prefix + "PUT /scenarios/{} HTTP/1.1\r\nContent-Length: ".format(str(scenario_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    name = await websocket.recv()
    parse_response(name)

# Give order to endpoint
async def get_device_data(websocket, id):
    # 10 here is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "GET /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n".format(str(id),str(id))
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)

######## Messages Logic

async def consumer(websocket):
    # Receiver
    while True:
        bytes_str = await websocket.recv()
        first = str(bytes_str[:20]) # Scanning 1st characters

        if ("PUT" in first):
            print('PUT message detected !')
            
            # print('RAW INCOMING :')
            # print(bytes_str)
            # print('END RAW')
            
            try:
                incoming = parse_put_response(bytes_str)
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                await parse_response(incoming)
                print('PUT message processed !')
            except:
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(incoming)
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")

        elif ("POST" in first):
            
            # print('RAW INCOMING :')
            # print(bytes_str)
            # print('END RAW')
            
            try:
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                incoming = parse_put_response(bytes_str)
                await parse_response(incoming)
                print('POST message processed !')
            except:
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print('RAW INCOMING :')
                print(bytes_str)
                print('END RAW')
            
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
        elif ("HTTP/1.1" in first): #(bytes_str != 0) and 
            response = response_from_bytes(bytes_str[len(cmd_prefix):])
            incoming = response.data.decode("utf-8")
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            # print(incoming)
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1)
            try:
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                await parse_response(incoming)
                print('GET response message processed !')
            except:
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(incoming)
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # await parse_put_response(incoming)
        else:
            print(bytes_str)
            
async def producer(websocket):
        # while True:
    await asyncio.sleep(48)
    # await get_ping(websocket)
    await get_data(tydom)
    print("Websocket refreshed at ", str(datetime.fromtimestamp(time.time())))
    

async def consumer_handler(websocket):
    while True:
        try:
            await consumer(websocket)
        except Exception as e:
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print('Consumer task has crashed !')
            print(e)
            print('Restarting..............')
            await main_task()

async def producer_handler(websocket):
    while True:
        await producer(websocket)

######## HANDLER
async def handler(websocket):
    try:
        # print("Starting handlers...")
        consumer_task = asyncio.ensure_future(
            consumer_handler(websocket))
        producer_task = asyncio.ensure_future(
            producer_handler(websocket))
        done, pending = await asyncio.wait(
            [consumer_task, producer_task],
            return_when=asyncio.FIRST_COMPLETED,
        )
        for task in pending:
            task.cancel()

    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print(e)
        print('Handler crashed.')


async def websocket_connection():
    global tydom
    # logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
    httpHeaders =  {"Connection": "Upgrade",
                    "Upgrade": "websocket",
                    "Host": host + ":443",
                    "Accept": "*/*",
                    "Sec-WebSocket-Key": generate_random_key(),
                    "Sec-WebSocket-Version": "13"
                    }
    # http.client.HTTPSConnection.debuglevel = 1
    # http.client.HTTPConnection.debuglevel = 1
    # Create HTTPS connection on tydom server
    conn = http.client.HTTPSConnection(host, 443, context=ssl_context)
    # Get first handshake
    conn.request("GET", "/mediation/client?mac={}&appli=1".format(mac), None, httpHeaders)
    res = conn.getresponse()
    # Get authentication
    nonce = res.headers["WWW-Authenticate"].split(',', 3)
    # read response
    res.read()
    # Close HTTPS Connection
    conn.close()
    # Build websocket headers
    websocketHeaders = {'Authorization': build_digest_headers(nonce)}
    if ssl_context is not None:
        websocket_ssl_context = ssl_context
    else:
        websocket_ssl_context = True # Verify certificate

    print('"Attempting websocket connection..."')
    ########## CONNECTION
    # websocket = await websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
    #                                      extra_headers=websocketHeaders, ssl=websocket_ssl_context)
    
    async with websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
                                         extra_headers=websocketHeaders, ssl=websocket_ssl_context) as websocket:
    
    
        tydom = websocket
        print("Tydom Websocket is Connected !", tydom)

        await mqttconnection(mqtt_host, mqtt_user, mqtt_pass)
        print("##################################")
        print('Requesting 1st data...')
        await get_data(tydom)
        while True:
            # await consumer(tydom) #Only receiving from socket in real time
            await handler(tydom) #If you want to send periodically something, disable consumer

# Main async task
async def main_task():
    print(str(datetime.fromtimestamp(time.time())))
    try:
        if (tydom == None) or not tydom.open:
            print("##################################")
            start = time.time()
            await websocket_connection()
            print('Connection total time :')
            end = time.time()
            print(end - start)

        else:
            print('Websocket is still opened ! requesting data...')
            await get_data(tydom)
    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print('Connection total time :')
        end = time.time()
        print(end - start)
        print(str(datetime.fromtimestamp(time.time())))
        print(e)
        print('Something bad happened, reconnecting...')
        # await asyncio.sleep(8)
        await main_task()


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    
    loop.run_until_complete(main_task())
    loop.run_forever()



        # Get informations (not very useful)
        # await get_info(websocket)

        # Get all moments stored on Tydom
        # await get_moments(websocket)

        # Get scenarios ids
        # print("Get scenarii")
        # await get_scenarios(websocket)

        # Run scenario with scn id returned in previous command
        # await put_scenarios(websocket, 15)
        # print("Get names of all devices")
        # await get_configs_file(websocket)
        # # await get_devices_meta(websocket)

        # print("Get data of all devices")
        # await get_devices_data(websocket)

        # Get data of a specific device
        #await get_device_data(websocket, 9)

        # Set a shutter position to 10%
        #await put_devices_data(websocket, 9, "position", "10.0")
        # TODO : Wait hardcoded for now to put response from websocket server
        #time.sleep(45)

[mrwiwi]

Exemple of logs from the last version 👍

2019-12-11 21:59:42.310451
##################################
"Attempting websocket connection..."
Tydom Websocket is Connected ! <websockets.client.WebSocketClientProtocol object at 0x039BD928>
Attempting MQTT connection...
##################################
Subscribing to :  homeassistant/+/tydom/#
##################################
Requesting 1st data...
MQTT is connected and suscribed ! =)
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
Incoming message type : config detected
Configuration updated
GET response message processed !
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:00:39.164854
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:01:29.154541
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'7'
GET response message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'12'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
PUT message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:02:19.163388
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'4'
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Websocket refreshed at  2019-12-11 22:03:09.157773
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Websocket refreshed at  2019-12-11 22:03:59.158697
Incoming message type : config detected
Configuration updated
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
b'\x02\x1e\x01\x00\x02\x00P\x02\x00\x14\xb1\xcc\x80\x00\x00\x82\x00\x00\x03\x00\x08\x00\x04\x0f\x00\x00\x01\xfdb\xfb|'  
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:04:49.159226
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:05:39.163886
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:06:29.167847
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Websocket refreshed at  2019-12-11 22:07:19.162555
Incoming message type : config detected
Configuration updated
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:08:09.185200
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:08:59.192184
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Alarm created / updated : alarm_tydom_1544791388
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:09:49.187374
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:10:39.207062
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:11:29.209469
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
PUT message processed !
PUT message detected !
Incoming message type : data detected
Alarm created / updated : alarm_tydom_1544791388
PUT message processed !
Websocket refreshed at  2019-12-11 22:12:19.202075
Incoming message type : config detected
Configuration updated
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:13:09.208218
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:13:59.208682
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !

[mrwiwi]

For the mqtt topics, "homeassistant" is the default prefix for hassio, but we can make it a variable.

It post config json for auto discovery, as it's documented here 👍
https://www.home-assistant.io/docs/mqtt/discovery/

[mrwiwi]

In the connection handler (producer and consumer), the producer peridcally send a get_data to the websocket to avoid disconnection when remote, i didn't succeded yet in the refresh/all POST as documented here : https://github.com/mgcrea/node-tydom-client

[cth35]

Ideally, it should also run on domoticz. gmqtt is not installed by default. I will have a look. Le mercredi 11 décembre 2019 à 21:44:51 UTC+1, WiwiWillou <notifications@github.com> a écrit : For the mqtt topics, "homeassistant" is the default prefix for hassio, but we can make it a variable. It post config json for auto discovery, as it's documented here 👍 https://www.home-assistant.io/docs/mqtt/discovery/ — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[mrwiwi]

The script just need to have the mqtt broker address, i run it from my hotel :)

I run hassio on docker on debian, so i will get it to run from debian. Probably the most client independant solution, we just need a broker (SSL via port 8883 in the script BTW)

[mrwiwi]

Does domoticz have auto discovery MQTT ?

[cth35]

yes, I think domoticz uses MQTT in its framework Le mercredi 11 décembre 2019 à 21:50:41 UTC+1, WiwiWillou <notifications@github.com> a écrit : Does domoticz have auto discovery MQTT ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[mrwiwi]

Fixed MQTT set position for covers !

async def on_message(client, topic, payload, qos, properties):
    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
    # print('MQTT incoming : ', topic, payload)
    if (topic == "homeassistant/requests/tydom/update") or (payload == "please"):
        await get_data(tydom)
    if ('set_position' in str(topic)):
        print('MQTT set_position incoming : ', topic, payload)
        get_id = (topic.split("/"))[3] #extract id from mqtt
        # print(tydom, str(get_id), 'position', json.loads(payload))
        await put_devices_data(tydom, str(get_id), 'position', str(json.loads(payload)))
    else:
        return 0

[mrwiwi]

TODO (IMHO) :

• Refresh POST type message
• ignore if values are flaged as expired, then refresh !
• Review of the code (i just learn python and programming for a few days...)
• Test with others systems
• Add more devices types
• MQTT topics for scenes/scenarii (set topics)
• Add attributes to alarm and covers (defects, etc.)

Thanks again :)

[cth35]

Domoticz suggests to use paho-mqtt for python.
For someone that learned python for a few days, you rock ;)

[mrwiwi]

Domoticz suggests to use paho-mqtt for python.
For someone that learned python for a few days, you rock ;)

Thanks that's very kind !

paho isn't async if i understand well, gmqtt works really well.

You launch the script from domoticz or by its side ?

Topics are topics, any library should work with any broker no ?

I've created a HA topic to recruit some more people, here : https://community.home-assistant.io/t/tydom2mqtt-make-delta-dore-devices-home-assistant-compatible/154439

It points to your github obviously !

[mrwiwi]

I have 1006 connection closed codes when on local, not an issue on remote !