Enhancement: Add MQTT WebSocket support for real-time temperature updates

[GarthDB]

🎯 Goal

Add MQTT over WebSocket support to provide real-time temperature updates (< 1 second latency) instead of 60-second REST API polling.

📊 Current Status

v0.1.0: REST API polling every 60 seconds ✅ Working perfectly
v0.2.0: MQTT real-time updates 🚧 Research complete, implementation needed

🔬 Reverse Engineering Complete

We've successfully reverse-engineered FireBoard's MQTT WebSocket protocol by analyzing browser WebSocket traffic:

Connection Details

URL: wss://fireboard.io/ws
Protocol: MQTT v3.1 over WebSocket (mqttv3.1)
Port: 443 (WSS)

Authentication

Method: Session cookies (NOT token in headers or URL)

Cookie: sessionid=xxx; csrftoken=yyy
Origin: https://fireboard.io
Sec-WebSocket-Protocol: mqttv3.1

Topic Structure

{device_uuid}/templog{channel_number}
{device_uuid}/drivelog

Examples:
- 9d6dbc3d-d53a-4924-b72b-f0167ab3ab4c/templog1
- 9d6dbc3d-d53a-4924-b72b-f0167ab3ab4c/templog2
- 9d6dbc3d-d53a-4924-b72b-f0167ab3ab4c/drivelog

Message Format (JSON)

{
  "temp": 67.0,
  "channel": 1,
  "p": true,              // probe present/connected
  "date": "2025-10-27 20:43:20 UTC",
  "degreetype": 2         // 2=Fahrenheit, 1=Celsius
}

Client ID Format

fireboard_web_{random_uuid}
Example: fireboard_web_c8adef7c-0b58-7f8e-2e59-6132c6050f6c

❌ Why Not Included in v0.1.0

We attempted implementation with paho-mqtt but encountered a critical issue:

The Problem

# Our attempt (doesn't work):
client = mqtt.Client(transport="websockets", protocol=mqtt.MQTTv31)
client.ws_set_options(
    path="/ws",
    headers={"Cookie": cookie_header}
)
client.connect("fireboard.io", 443)
# Result: WebSocket handshake error, connection not upgraded

Root Cause: paho-mqtt's WebSocket implementation doesn't properly support:

1. Session cookie authentication in WebSocket handshake
2. The specific WebSocket upgrade protocol FireBoard's server expects

✅ Implementation Plan

Option 1: Custom WebSocket + MQTT Implementation (Recommended)

Use websockets library with manual MQTT packet handling:

import websockets
import struct

class FireBoardMQTTClient:
    async def connect(self, session_cookies: dict):
        headers = {
            'Cookie': '; '.join([f'{k}={v}' for k, v in session_cookies.items()]),
            'Origin': 'https://fireboard.io',
            'Sec-WebSocket-Protocol': 'mqttv3.1',
        }
        
        self.ws = await websockets.connect(
            'wss://fireboard.io/ws',
            extra_headers=headers
        )
        
        # Send MQTT CONNECT packet
        await self._send_mqtt_connect()
        
        # Subscribe to topics
        for device_uuid, channels in devices:
            for channel in channels:
                await self._send_mqtt_subscribe(f"{device_uuid}/templog{channel}")
    
    async def _send_mqtt_connect(self):
        # Build MQTT v3.1 CONNECT packet
        # See: http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html
        client_id = f"fireboard_ha_{self.token[:8]}"
        # ... packet construction
        await self.ws.send(packet)
    
    async def _send_mqtt_subscribe(self, topic: str):
        # Build MQTT SUBSCRIBE packet
        # ... packet construction
        await self.ws.send(packet)
    
    async def _receive_loop(self):
        async for message in self.ws:
            # Parse MQTT packet
            packet_type = message[0] >> 4
            if packet_type == 3:  # PUBLISH
                topic, payload = self._parse_mqtt_publish(message)
                # Parse JSON payload
                data = json.loads(payload)
                self.on_message_callback(topic, data)

Required Dependencies

# Add to manifest.json requirements:
"websockets>=12.0"

Files to Modify

1. custom_components/fireboard/mqtt_client.py - Replace paho-mqtt with custom implementation
2. custom_components/fireboard/coordinator.py - Update MQTT setup
3. custom_components/fireboard/manifest.json - Update dependencies and iot_class
4. tests/test_mqtt_client.py - Update tests for new implementation

MQTT Packet Format References

• MQTT v3.1 Spec: http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html
• CONNECT packet: Fixed header (0x10) + Variable header (protocol name, version, flags) + Payload (client ID)
• SUBSCRIBE packet: Fixed header (0x82) + Variable header (packet ID) + Payload (topic filters)
• PUBLISH packet: Fixed header (0x30) + Variable header (topic name) + Payload (message)

Testing Approach

# Mock WebSocket for unit tests
class MockWebSocket:
    async def send(self, data):
        # Verify MQTT packet structure
        assert data[0] == 0x10  # CONNECT packet
    
    async def recv(self):
        # Return mock MQTT PUBLISH packet
        return self._build_mock_publish(topic, payload)

# Integration test with real FireBoard
@pytest.mark.integration
async def test_mqtt_real_connection():
    client = FireBoardMQTTClient(token, cookies)
    await client.connect()
    # Wait for temperature message
    msg = await asyncio.wait_for(client.receive(), timeout=10)
    assert 'temp' in msg

📈 Benefits

Feature	REST Polling (v0.1.0)	MQTT Push (v0.2.0)
Update Latency	60 seconds	< 1 second
API Calls/Hour	60 per device	~5 per device
Battery Impact	Higher (frequent polls)	Lower (push only)
Rate Limit Risk	Low	Minimal
Real-time Alerts	❌ Delayed	✅ Instant

📚 Reference Documentation

All research is documented in:

• analyze_websocket.md - WebSocket capture instructions
• MQTT_IMPLEMENTATION.md - Original MQTT plan (needs update)
• Code comments in mqtt_client.py - Current paho-mqtt attempt

Browser DevTools Capture

The WebSocket traffic was captured showing:

• MQTT CONNECT with MQIsdp protocol header
• Client ID format
• SUBSCRIBE packets for each templog channel
• PUBLISH packets with JSON temperature data

🧪 Testing Requirements

1. Unit Tests

   • MQTT packet construction/parsing
   • WebSocket connection handling
   • Message callback functionality

2. Integration Tests

   • Connect to real FireBoard WebSocket
   • Subscribe to topics
   • Receive temperature updates
   • Handle disconnection/reconnection

3. Real-World Testing

   • Multiple devices simultaneously
   • Long-running connection stability
   • Network interruption recovery
   • Temperature update accuracy

🎯 Acceptance Criteria

• WebSocket connection established with session cookies
• MQTT v3.1 packets properly constructed and sent
• Successfully subscribe to device topics
• Temperature updates received in < 1 second
• JSON payload correctly parsed
• Sensor values updated in Home Assistant
• Graceful reconnection on disconnect
• All unit tests passing
• Integration tests with real FireBoard passing
• Documentation updated

🔄 Migration Path

The integration should:

1. Attempt MQTT connection on setup
2. Fall back to REST polling if MQTT fails
3. Log warnings but continue functioning
4. Allow users to disable MQTT via config option

This ensures backward compatibility and reliability.

📝 Notes

• FireBoard's MQTT broker is not publicly documented
• Implementation based on reverse engineering browser WebSocket traffic
• Protocol matches standard MQTT v3.1 over WebSocket
• Session cookies are critical for authentication
• No username/password authentication on MQTT layer

📅 Target

Milestone: v0.2.0
Priority: Medium (v0.1.0 REST polling is fully functional)
Complexity: High (custom MQTT implementation required)
Estimated Effort: 2-3 days development + testing