Skip to content

MohiCodeHub/airbio-track

BranchesTags

Repository files navigation

Arbio - AI Property Onboarding & Maintenance Platform

An AI-powered system for property managers that enables:

  • Property Onboarding: Walk through properties with your phone camera, AI extracts all listing information
  • Maintenance Tracking: Housekeepers send videos via WhatsApp, AI detects issues and alerts property managers
  • Guest Guidebook: Voice-first interface for guests to ask questions about the property

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                              ARBIO PLATFORM                                  │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                   │
│  │   Frontend   │    │  WhatsApp    │    │   Backend    │                   │
│  │   (React)    │◄──►│     Bot      │◄──►│   Camera     │                   │
│  │   :5173      │    │   (Node)     │    │   (Flask)    │                   │
│  └──────────────┘    │   :3000      │    │   :5000      │                   │
│         │            └──────────────┘    └──────────────┘                   │
│         │                   │                   │                            │
│         ▼                   ▼                   ▼                            │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                   │
│  │  Extraction  │    │  VideProc    │    │   MongoDB    │                   │
│  │   Service    │    │   Service    │    │              │                   │
│  │   (FastAPI)  │    │  (FastAPI)   │    │   :27017     │                   │
│  │   :8000      │    │   :8001      │    │              │                   │
│  └──────────────┘    └──────────────┘    └──────────────┘                   │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

Services

Service Description Port Tech Stack
Frontend Property onboarding UI, dashboard, guest guidebook 5173 React, Vite, TypeScript, Tailwind
Backend Camera Real-time video analysis, property inspection 5000 Flask, OpenCV, OpenRouter API
Extraction Service Document parsing, AI data extraction from PDFs/images 8000 FastAPI, GPT-4o, PyMuPDF
VideProc Service Video analysis for maintenance, Gmail notifications 8001 FastAPI, OpenAI, Gmail API
WhatsApp Bot Receives videos from housekeepers via WhatsApp 3000 Node.js, whatsapp-web.js
MongoDB Database for maintenance logs, property data 27017 MongoDB

Quick Start with Docker

The easiest way to run all services:

# 1. Clone the repository
git clone <repo-url>
cd TEHack

# 2. Copy environment template and configure
cp .env.example .env
# Edit .env with your API keys

# 3. Start all services
docker-compose up -d

# 4. Check status
docker-compose ps

# 5. View logs
docker-compose logs -f

Access Points

Manual Setup (Development)

Prerequisites

  • Node.js 18+ (for frontend and WhatsApp bot)
  • Python 3.10+ (for backend services)
  • MongoDB (local or cloud instance)
  • API Keys:
    • OpenRouter API key (for vision models)
    • OpenAI API key (for GPT-4o extraction)
    • Gmail OAuth credentials (for PM notifications)

1. Frontend (React + Vite)

The main user interface for property onboarding and dashboard.

cd frontend

# Install dependencies
npm install

# Start development server
npm run dev

Available Scripts:

npm run dev      # Start dev server (http://localhost:5173)
npm run build    # Production build
npm run preview  # Preview production build
npm run lint     # Run ESLint
npm run test     # Run tests

Environment Variables: Create frontend/.env:

VITE_API_URL=http://localhost:5000
VITE_EXTRACTION_API_URL=http://localhost:8000

2. Backend Camera (Flask)

Real-time video analysis using AI vision models via OpenRouter.

cd backend-camera

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Set API key
export OPENROUTER_API_KEY="your-api-key-here"

# Run the server
python app.py

Available Commands:

python app.py              # Web UI on http://localhost:5000
python main.py             # CLI mode (headless)
python main.py --help      # Show CLI options
python main.py --source 0  # Use specific webcam
python main.py --fps 2     # Analyze 2 frames per second

Environment Variables:

OPENROUTER_API_KEY=your-openrouter-api-key
MONGODB_URI=mongodb://localhost:27017
MONGODB_DB_NAME=arbio

3. Extraction Service (FastAPI)

AI-powered document parsing and property data extraction.

cd services/extraction

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Set API key
export OPENAI_API_KEY="your-openai-api-key"

# Run the server
uvicorn app.main:app --reload --port 8000

API Endpoints:

Endpoint Method Description
/ GET Health check
/extract/pdf POST Extract data from PDF
/extract/image POST Extract data from image
/extract/text POST Extract data from text
/schema GET Get property schema
/properties GET List all properties
/properties/{id} GET Get property details
/properties/{id}/gaps GET Get missing fields
/properties/{id}/draft-email POST Generate follow-up email

API Documentation: http://localhost:8000/docs

4. VideProc Service (FastAPI)

Video analysis for maintenance detection with Gmail notifications.

cd videoproc

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export OPENAI_API_KEY="your-openai-api-key"
export MONGODB_URI="mongodb://localhost:27017"
export PM_EMAIL="pm@example.com"

# Run the server
uvicorn src.main:app --reload --port 8001

Gmail OAuth Setup:

  1. Create a project in Google Cloud Console
  2. Enable Gmail API
  3. Create OAuth 2.0 credentials
  4. Download credentials.json to videoproc/
  5. Run the token refresh script:
python scripts/get_refresh_token.py

API Endpoints:

Endpoint Method Description
/ GET Health check
/webhook/whatsapp POST Receive video from WhatsApp bot
/logs GET Get maintenance logs
/logs/{id} GET Get specific log
/videos GET List uploaded videos
/poll-pm-replies GET Check for PM email responses

5. WhatsApp Bot (Node.js)

Receives videos from housekeepers and forwards to VideProc service.

cd whatsapp-bot

# Install dependencies
npm install

# Create .env file
cp .env.example .env
# Edit .env with your settings

# Start the bot
npm start

First-Time Setup:

  1. Run npm start
  2. Scan the QR code with WhatsApp on your phone
  3. Go to Settings > Linked Devices > Link a Device
  4. Session is saved for future restarts

Environment Variables: Create whatsapp-bot/.env:

# Required: VideProc endpoint
VIDEO_ENDPOINT_URL=http://localhost:8001/webhook/whatsapp

# Optional: API authentication
API_KEY=your-api-key

# Optional: Restrict to specific phone numbers
ALLOWED_NUMBERS=1234567890,0987654321

# Optional: Logging level
LOG_LEVEL=info

Commands (send via WhatsApp):

  • !help - Show help
  • !status - Check bot status

6. MongoDB

Start MongoDB locally or use MongoDB Atlas.

Local (Docker):

docker run -d --name mongodb -p 27017:27017 mongo:7

Local (Homebrew on macOS):

brew services start mongodb-community

MongoDB Atlas (Cloud):

  1. Create free cluster at https://cloud.mongodb.com
  2. Get connection string
  3. Set MONGODB_URI in your .env files

Environment Variables Reference

Create a .env file in the root directory:

# ===== API Keys =====
OPENROUTER_API_KEY=sk-or-v1-xxxxx
OPENAI_API_KEY=sk-xxxxx

# ===== MongoDB =====
MONGODB_URI=mongodb://localhost:27017
MONGODB_DB_NAME=arbio

# ===== Services =====
VITE_API_URL=http://localhost:5000
VITE_EXTRACTION_API_URL=http://localhost:8000
VIDEO_ENDPOINT_URL=http://localhost:8001/webhook/whatsapp

# ===== Gmail (for PM notifications) =====
PM_EMAIL=pm@example.com
GMAIL_CLIENT_ID=xxxxx.apps.googleusercontent.com
GMAIL_CLIENT_SECRET=xxxxx
GMAIL_REFRESH_TOKEN=xxxxx

# ===== WhatsApp Bot =====
ALLOWED_NUMBERS=
LOG_LEVEL=info

Docker Compose Services

# Start all services
docker-compose up -d

# Start specific services
docker-compose up -d frontend backend-camera

# View logs
docker-compose logs -f extraction

# Stop all services
docker-compose down

# Rebuild after code changes
docker-compose up -d --build

Development Workflow

Running Tests

# Frontend tests
cd frontend && npm run test

# Python services (if tests exist)
cd services/extraction && python -m pytest

Code Structure

TEHack/
├── frontend/                 # React frontend application
│   ├── src/
│   │   ├── components/      # UI components
│   │   ├── pages/           # Page components
│   │   ├── lib/             # Utilities, API clients
│   │   └── hooks/           # React hooks
│   └── package.json
│
├── backend-camera/          # Flask vision analysis service
│   ├── app.py               # Web UI entry point
│   ├── main.py              # CLI entry point
│   ├── camera.py            # Video capture
│   ├── vision.py            # AI analysis
│   └── config.py            # Configuration
│
├── services/
│   └── extraction/          # FastAPI extraction service
│       ├── app/
│       │   ├── main.py      # API entry point
│       │   ├── extractors/  # PDF, image, text extractors
│       │   ├── agent/       # Gap detection, email drafting
│       │   └── models/      # Data models
│       └── requirements.txt
│
├── videoproc/               # FastAPI video processing service
│   ├── src/
│   │   ├── main.py          # API entry point
│   │   ├── services/        # Video analysis
│   │   └── config/          # Gmail OAuth
│   └── requirements.txt
│
├── whatsapp-bot/            # Node.js WhatsApp bot
│   ├── src/
│   │   ├── index.js         # Bot entry point
│   │   └── config.js        # Configuration
│   └── package.json
│
├── docker/                  # Dockerfiles for each service
├── docker-compose.yml       # Orchestration
└── README.md                # This file

Troubleshooting

Common Issues

OpenRouter API Key not working:

# Verify key is set
echo $OPENROUTER_API_KEY

# Test the API
curl -H "Authorization: Bearer $OPENROUTER_API_KEY" https://openrouter.ai/api/v1/models

MongoDB connection failed:

# Check if MongoDB is running
docker ps | grep mongo

# Or check local service
brew services list | grep mongodb

WhatsApp QR code not appearing:

  • Ensure terminal supports QR codes
  • Delete .wwebjs_auth/ folder and restart

Video analysis slow:

  • Reduce FPS in backend-camera/config.py
  • Use faster models like gemini-1.5-flash
  • Reduce MAX_FRAME_DIMENSION

License

MIT License - See individual service folders for specific licenses.

Contributors

Built at the Berlin Hackathon 2025

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages