Device Linking

Device Linking & Distributed Computing

Status: Core implemented ✅ (QR pairing, WSS/TLS, BLE auto-discovery — v0.5.0)

Core Concept

Zelara processes tasks on-edge (locally) with device linking for distributed computing across user's devices.

Processing Priority: Desktop > Mobile > Web

When a task requires more compute than current device can handle, offload to most capable linked device.

Use Cases

Example 1: Recycling Image Validation

1. User takes photo on Mobile of paper bag with recyclables
2. Mobile detects it can't run full CV model (battery/performance constraints)
3. Mobile sends image to linked Desktop for processing
4. Desktop runs CV validation, returns result to Mobile
5. Mobile displays validation result, awards points

Example 2: Carbon Footprint Calculation

1. User inputs property data on Mobile (house type, location, energy usage)
2. Mobile detects complex calculation required (local regulations, solar viability, etc.)
3. Mobile offloads to linked Desktop
4. Desktop runs calculations with full datasets, generates reduction pathway
5. Desktop sends results back to Mobile
6. Mobile displays personalized carbon reduction plan

Example 3: Tax Preparation Analysis

1. User logs expenses throughout day on Mobile
2. At night, Desktop (if linked) automatically processes accumulated data
3. Desktop categorizes expenses, applies tax rules, updates deductions
4. Next morning, Mobile shows updated tax prep status

Architecture Questions

Device Discovery

• How do devices find each other?
  • Option A: Local network discovery (mDNS/Bonjour)
  • Option B: Bluetooth LE for proximity pairing
  • Option C: Cloud relay as fallback (requires backend)
  • Option D: Manual pairing via QR code
  • Hybrid: Local-first (A/B), cloud relay only if needed (C)

Task Offloading Protocol

• How does app decide to offload?

  • Device capability detection (CPU, RAM, battery level)
  • Task complexity assessment
  • User preference (always offload vs only when necessary)

• How are tasks sent?

  • Direct device-to-device (local network)
  • End-to-end encrypted messages
  • Binary protocol vs JSON

State Synchronization

• What gets synced across devices?

  • User unlock state (which features unlocked)
  • Task completion history
  • Points/progress
  • Module preferences
  • Application data (finances, recycling logs, etc.)

• When does sync happen?

  • Real-time (as changes occur)
  • Periodic (every X minutes)
  • On-demand (user triggers)
  • Event-driven (after task completion, feature unlock, etc.)

Security & Privacy

• How to secure device linking?

  • Asymmetric encryption (each device has key pair)
  • Device approval flow (user confirms on both devices)
  • Device revocation (unlink devices)
  • Temporary pairing vs permanent linking

• How to protect data in transit?

  • End-to-end encryption for all device communication
  • Images never leave user's devices unencrypted
  • No cloud intermediary sees plaintext data

Fallback Handling

• What if no capable device linked?

  • Option A: Prompt user to link capable device
  • Option B: Degrade functionality (simpler algorithm, lower accuracy)
  • Option C: Queue task for later (process when Desktop available) ← Preferred for MVP
  • Option D: Optional cloud processing (defeats edge-first principle)

• What if linked device is offline?

  • Queue task locally (save to AsyncStorage with photo data)
  • Wait for device to come online (background connection checks)
  • Auto-sync when Desktop available (process queued tasks automatically)
  • Timeout and fallback to local processing (degraded) - deferred to v2

Technology Options

Device Communication

• WebRTC - P2P communication, NAT traversal, works across platforms
• gRPC - Efficient binary protocol, cross-platform, good for task offloading
• WebSockets - Real-time bidirectional, requires intermediary server
• Custom UDP/TCP - Direct control, platform-specific implementation needed

Service Discovery

• mDNS/Bonjour - Local network discovery, native iOS/macOS support
• Bluetooth LE - Proximity-based, low power, good for mobile
• Manual pairing - QR code, simplest implementation, no automatic discovery

State Sync

• CRDTs - Conflict-free replicated data types, eventual consistency
• Operational Transform - Real-time sync, complex implementation
• Last-write-wins - Simple, potential data loss
• Manual conflict resolution - User chooses which device wins

MVP Scope

For MVP, we need to decide:

1. Include device linking in MVP or defer to v2?

   • If included: Which use case? (Image validation most compelling)
   • If deferred: How does MVP handle compute-heavy tasks? (Lighter models? Desktop-only?)

2. Simplest device linking approach?

   • Manual pairing via QR code
   • Single task offload (image validation)
   • No automatic sync, just on-demand task processing

3. Security minimum?

   • Basic encryption (HTTPS/TLS)
   • Device approval flow
   • Defer complex key management to v2

Open Questions

1. Should web version support device linking? (Or only Desktop ↔ Mobile?)
2. Can multiple Desktops be linked? (Home desktop + work laptop?)
3. Battery/network awareness for offloading decisions?
4. User visibility into task offloading? (Silent or show "Processing on Desktop..."?)
5. Offline-first: How long to queue tasks before failing?

Mobile Offline Architecture (Future Enhancement)

Status: Planned - Not yet implemented

Current Limitation

MVP implementation requires Desktop to be available when Mobile tasks are submitted. If Desktop is offline, tasks fail immediately.

Goal: Fully Offline Mobile Operation

Enable this workflow:

1. 📱 Mobile offline (no Desktop connection) → User takes photos, completes tasks
2. 💾 Tasks queued locally on Mobile (stored in AsyncStorage)
3. 💻 Desktop opens + Zelara runs → Mobile auto-detects and connects
4. 🔄 Queued tasks sync to Desktop → Desktop processes (AI/CV) → Results back to Mobile
5. ✅ Points awarded, modules unlocked retroactively

Key Clarification: Metro is Development-Only

Common confusion: When testing Mobile with Metro bundler + Android Studio, the app requires WiFi connection to laptop.

Reality:

• Development mode (Metro): JS code served real-time from laptop → requires WiFi
• Production mode (Release APK): JS bundle embedded in APK → fully offline

How it works:

1. Build release APK: npm run bundle:android && ./gradlew assembleRelease
2. APK contains index.android.bundle in assets directory
3. App loads bundle from local filesystem at launch
4. No Metro, no WiFi, no laptop required ✅

Mobile app runs completely standalone. Device linking is only for offloading heavy compute tasks (AI/CV), not for basic app operation.

Architecture Components

1. Task Queue Service (TaskQueueService.ts)

// Pseudo-implementation
class TaskQueueService {
  async enqueueTask(task: Task): Promise<void>
  async getQueuedTasks(): Promise<Task[]>
  async markTaskProcessed(taskId: string): Promise<void>
  async clearQueue(): Promise<void>
}

interface Task {
  id: string
  type: 'image_validation' | 'calculation' | 'analysis'
  payload: any // Image data, input parameters, etc.
  timestamp: string
  status: 'pending' | 'processing' | 'completed' | 'failed'
  retries: number
}

Storage: AsyncStorage (React Native) for persistence across app restarts

Behavior:

• When DeviceLinkingService.sendImageValidation() fails (Desktop offline)
• Catch error → Save task to queue
• Show user: "Saved for processing when Desktop available"

2. Sync Service (SyncService.ts)

class SyncService {
  async checkDesktopAvailability(): Promise<boolean>
  async syncQueuedTasks(): Promise<SyncResult>
  startBackgroundSync(): void // Periodic connection checks
  stopBackgroundSync(): void
}

Behavior:

• On app launch: Check if Desktop is reachable (last paired IP/port)
• Background checks: Every 30 seconds when app active
• On Desktop detected: Automatically process queued tasks
• Notify user when sync completes: "Processed 5 tasks, earned 50 points!"

3. Connection State Persistence

Store in AsyncStorage:

{
  "lastPairedDesktop": {
    "ip": "192.168.1.100",
    "port": 8765,
    "token": "encrypted_pairing_token",
    "lastConnected": "2026-02-24T10:30:00Z"
  }
}

Use for:

• Auto-reconnect on app launch
• Background connection health checks
• Skip pairing flow if Desktop already linked

4. UI Indicators

HomeScreen additions:

• Desktop connection status: 🟢 Connected | 🟡 Connecting... | ⚪ Offline
• Queue indicator: "3 tasks pending" (when Desktop offline)
• Sync notification: Toast when queued tasks complete

Implementation Phases

Phase 1: Verify Release Build Works Offline

• Build release APK with embedded bundle
• Install on physical device (no laptop connection)
• Test basic app functionality
• Confirms: Mobile doesn't need WiFi for core operation ✅

Phase 2: Task Queueing

• Implement TaskQueueService
• Update RecyclingTaskScreen to queue tasks on Desktop unavailable
• Store photos as base64 or file references in AsyncStorage
• Update ProgressService to handle deferred point awards

Phase 3: Auto-Sync

• Implement SyncService
• Background connection checks (every 30s when app active)
• Auto-process queued tasks when Desktop detected
• Notify user on completion

Phase 4: Connection State Persistence

• Store last paired Desktop in AsyncStorage
• Auto-reconnect on app launch
• Skip QR pairing if Desktop already linked

Phase 5: UI Polish

• Connection status indicator
• Queue count badge
• Sync progress notifications
• Error handling and retry logic

Open Questions

1. Queue size limits: Max tasks to store? Oldest-first eviction?
2. Photo storage: Base64 in AsyncStorage vs file references? Compression?
3. Retry logic: How many retries before marking task failed?
4. Battery awareness: Pause background sync when battery low?
5. Conflict resolution: If user re-submits same task while queued?
6. Offline indicators: Show which tasks are queued vs completed?

Dependencies

Requires from MVP:

• ✅ Device linking protocol (QR pairing, WebSocket)
• ✅ Task offloading (image validation)
• ✅ AsyncStorage integration

New dependencies:

• Background task scheduling (React Native background jobs)
• Connection health checks (periodic ping to Desktop)
• Queue management logic

Decisions Needed

• Device discovery mechanism → QR pairing (MVP) + BLE auto-discovery (v0.5.0)
• Task offloading protocol → WebSocket with JSON (MVP)
• State sync strategy → Deferred to offline queue enhancement
• Security model → TLS encryption, pairing tokens (QR); proximity trust (BLE)
• Fallback handling approach → Queue tasks when Desktop offline (Enhancement)
• MVP scope: Include device linking or defer? → Included in MVP

Decisions Log

2026-02-24 - Offline Queue Architecture Designed

Decision: Implement task queueing as post-MVP enhancement

Rationale:

• MVP proves device linking works (when Desktop available)
• Offline queueing adds complexity but huge UX improvement
• Breaking into phases allows iterative implementation
• Production APK already works offline (no Metro dependency)

Next steps:

1. MVP ships with "Desktop required" limitation
2. Post-MVP: Add task queue + auto-sync
3. Document thoroughly for future implementation

2026-03-14 - BLE Auto-Discovery Implemented (v0.5.0)

Decision: Implement BLE-based auto-discovery as Windows-first feature in v1

Rationale:

• Eliminates QR scan friction for the common case (both devices at home)
• BLE is discovery-only — WSS/TLS data path unchanged
• WinRT APIs available without extra native modules on Desktop
• react-native-ble-plx handles Android scanning cleanly
• macOS/Linux pluggable later via #[cfg(target_os)] guards

Implementation:

• Desktop: ble_advertising.rs — WinRT BluetoothLEAdvertisementPublisher; service UUID a1b2c3d4-e5f6-7a8b-9c0d-e1f2a3b4c5d6; manufacturer data = [0xFE, 0xFF, ip0, ip1, ip2, ip3, port_hi, port_lo]
• Mobile: BLEDiscoveryService.ts — scans for service UUID, parses IP:port, calls DeviceLinkingService.connect() with discoveryMethod: 'ble'
• Security: BLE connections skip token validation (proximity = trust); is_ble_connection session flag is sticky for whole session
• QR pairing remains fully functional as fallback

---

This document will be updated as decisions are made and architecture solidifies.