Audio Event Monitor for Raspberry Pi

Real-time audio classification system that detects and logs important sound events (Alarms, Clanging, Help, Movement) while filtering out background noise. Built with TensorFlow Lite for efficient edge deployment.

🎯 Features

• Real-time audio monitoring with TensorFlow Lite
• 80% confidence threshold for high-accuracy detection
• Background noise filtering (not logged to database)
• SQLite database for local event storage
• Optional notifications: Email, SMS (Twilio), Discord, Slack, Webhooks, Firebase
• Configurable alerts with cooldown periods
• Cross-platform: Test on PC, deploy to Raspberry Pi

📋 System Requirements

For Development (Windows/Mac/Linux)

• Python 3.8+
• Microphone
• TensorFlow 2.8+ (full version)

For Deployment (Raspberry Pi)

• Raspberry Pi 3B+ or newer
• Raspberry Pi OS (Bullseye or newer)
• USB Microphone
• Python 3.8+
• tflite_runtime (lightweight)

📁 Project Structure

ece-198/
├── raspberry_pi_monitor.py          # Main monitoring script
├── test_model.py                    # Test TFLite model
├── view_database.py                 # Query/view database
├── config.py                        # Configuration & API keys
├── converted_tflite/
│   ├── soundclassifier_with_metadata.tflite  # TFLite model
│   └── labels.txt                   # Classes: Alarms, Background Noise, Clanging, Help, Movement
├── tm-my-audio-model/               # Original Teachable Machine model (TF.js)
│   ├── model.json
│   ├── metadata.json
│   └── weights.bin
├── audio_events.db                  # SQLite database (auto-created)
├── requirements.txt                 # PC dependencies
├── requirements_pi.txt              # Raspberry Pi dependencies
├── RASPBERRY_PI_SETUP.md           # Complete Pi deployment guide
└── README.md                        # This file

🚀 Quick Start

1. Clone Repository

git clone https://github.com/KerelosGayed/ece-198.git
cd ece-198

2. Set Up Python Environment

On PC (Windows):

# Create virtual environment
python -m venv venv

# Activate venv
.\venv\Scripts\Activate.ps1

# Install dependencies
pip install tensorflow sounddevice scipy requests

On Raspberry Pi:

# Install system dependencies
sudo apt-get update
sudo apt-get install -y portaudio19-dev python3-pip

# Install Python packages
pip3 install --extra-index-url https://google-coral.github.io/py-repo/ tflite_runtime
pip3 install -r requirements_pi.txt

3. Configure Settings

Edit config.py to customize:

# Detection settings
CONFIDENCE_THRESHOLD = 0.80  # 80% minimum confidence
SAMPLE_RATE = 44100          # Audio sample rate
AUDIO_DURATION = 1.0         # Seconds per clip

# Alert settings
ALERT_ON_EVENTS = ["Help", "Alarms"]  # Only these trigger alerts
ALERT_CONFIDENCE_THRESHOLD = 0.85      # 85% for alerts

# Add API keys for notifications (optional)
EMAIL_ENABLED = False
SMS_ENABLED = False
DISCORD_ENABLED = False
# See config.py for full configuration options

4. Test Model

# On PC (with venv activated):
python test_model.py

# On Raspberry Pi:
python3 test_model.py

5. Run Monitor

On PC:

.\venv\Scripts\Activate.ps1
python raspberry_pi_monitor.py

On Raspberry Pi:

python3 raspberry_pi_monitor.py

6. View Results

# View recent events
python view_database.py recent 20

# View statistics
python view_database.py summary

# Export to CSV
python view_database.py export events.csv

📊 How It Works

Event Detection Flow

1. Audio Capture: 1-second audio clips at 44.1kHz
2. Preprocessing: Normalize raw audio to model input format [1, 44032]
3. Inference: TFLite model classifies audio into 5 categories
4. Filtering:
   • Requires ≥80% confidence
   • Background Noise is ignored (not logged)
5. Logging: Events saved to SQLite database with timestamp
6. Alerts: Optional notifications via Email/SMS/Discord/Slack/Webhooks

Detected Events

Event	Description	Logged to DB	Can Trigger Alerts
Alarms	Alarm sounds	✅ Yes	✅ Yes
Clanging	Metal/object impacts	✅ Yes	✅ Yes
Help	Voice calling for help	✅ Yes	✅ Yes (Default)
Movement	Physical movement sounds	✅ Yes	✅ Yes
Background Noise	Ambient/general noise	❌ Filtered	❌ No

Database Schema

Table: audio_events

Column	Type	Description
id	INTEGER	Auto-increment primary key
timestamp	TEXT	ISO 8601 format
event_type	TEXT	Alarms, Clanging, Help, or Movement
confidence	REAL	0.0 - 1.0 (model confidence)
all_predictions	TEXT	Full breakdown of all class predictions

Example Row:

id: 1
timestamp: 2025-11-10T23:37:24.859284
event_type: 3 Help
confidence: 0.9492
all_predictions: 0 Alarms: 2.14%, 1 Background Noise: 1.52%, 2 Clanging: 0.98%, 3 Help: 94.92%, 4 Movement: 0.44%

🔧 Configuration Guide

Basic Settings (config.py)

# Detection thresholds
CONFIDENCE_THRESHOLD = 0.80        # 80% minimum to log
ALERT_CONFIDENCE_THRESHOLD = 0.85  # 85% minimum to alert

# Audio configuration  
SAMPLE_RATE = 44100                # Hz (match model training)
AUDIO_DURATION = 1.0               # Seconds per clip
AUDIO_CHANNELS = 1                 # Mono
AUDIO_DEVICE_INDEX = None          # None = default mic

# Database
DATABASE_PATH = "audio_events.db"

# Model paths
MODEL_PATH = "converted_tflite/soundclassifier_with_metadata.tflite"
LABELS_PATH = "converted_tflite/labels.txt"

# Display options
VERBOSE = True                     # Show all detections

Alert Configuration

# Which events trigger alerts?
ALERT_ON_EVENTS = ["Help", "Alarms"]  # Empty [] = all events

# Cooldown prevents alert spam
ALERT_COOLDOWN = 60  # Seconds between alerts for same event type

Email Notifications (Gmail)

EMAIL_ENABLED = True
EMAIL_SMTP_SERVER = "smtp.gmail.com"
EMAIL_SMTP_PORT = 587
EMAIL_FROM = "your-email@gmail.com"
EMAIL_PASSWORD = "your-app-password"  # Not regular password!
EMAIL_TO = ["recipient1@gmail.com", "recipient2@gmail.com"]

Get Gmail App Password:

1. Go to https://myaccount.google.com/apppasswords
2. Select "Mail" and generate password
3. Use generated 16-character password (not your regular Gmail password)

SMS Notifications (Twilio)

SMS_ENABLED = True
TWILIO_ACCOUNT_SID = "ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
TWILIO_AUTH_TOKEN = "your_auth_token"
TWILIO_FROM_NUMBER = "+15551234567"
TWILIO_TO_NUMBERS = ["+15559876543", "+15551111111"]

Get Twilio Credentials:

1. Sign up at https://www.twilio.com/
2. Get Account SID and Auth Token from dashboard
3. Purchase a phone number or use trial number

Discord Webhook

DISCORD_ENABLED = True
DISCORD_WEBHOOK_URL = "https://discord.com/api/webhooks/123456789/abcdefgh..."

Create Discord Webhook:

1. Server Settings → Integrations → Webhooks
2. Click "New Webhook"
3. Copy webhook URL

Slack Webhook

SLACK_ENABLED = True
SLACK_WEBHOOK_URL = "https://hooks.slack.com/services/T00000000/B00000000/XXXX"

Create Slack Webhook:

1. Go to https://api.slack.com/messaging/webhooks
2. Create new incoming webhook
3. Copy webhook URL

Custom Webhook

WEBHOOK_ENABLED = True
WEBHOOK_URL = "https://your-server.com/api/audio-events"
WEBHOOK_SECRET = "your-secret-key"  # Optional authentication

Webhook POST Format:

{
  "event_type": "3 Help",
  "confidence": 0.9492,
  "timestamp": "2025-11-10T23:37:24.859284",
  "predictions": {
    "0 Alarms": 0.0214,
    "1 Background Noise": 0.0152,
    "2 Clanging": 0.0098,
    "3 Help": 0.9492,
    "4 Movement": 0.0044
  }
}

Firebase Firestore

FIREBASE_ENABLED = True
FIREBASE_CREDENTIALS_PATH = "firebase-credentials.json"
FIREBASE_COLLECTION = "audio_events"

Setup Firebase:

1. Go to https://console.firebase.google.com/
2. Create project → Firestore Database
3. Project Settings → Service Accounts → Generate new private key
4. Save JSON file as firebase-credentials.json

🎤 Audio Device Selection

List Available Devices

# On PC:
python -c "import sounddevice as sd; print(sd.query_devices())"

# On Raspberry Pi:
python3 -c "import sounddevice as sd; print(sd.query_devices())"

Select Specific Device

Edit config.py:

AUDIO_DEVICE_INDEX = 2  # Use device index from list above

📊 Viewing Data

Command Line

# Show 20 most recent events
python view_database.py recent 20

# Show summary statistics
python view_database.py summary

# Show events from last hour
python view_database.py hour

# Export to CSV
python view_database.py export my_events.csv

Direct Database Query

# Open database
sqlite3 audio_events.db

# SQL queries
SELECT * FROM audio_events ORDER BY timestamp DESC LIMIT 10;
SELECT event_type, COUNT(*) FROM audio_events GROUP BY event_type;
SELECT * FROM audio_events WHERE confidence > 0.9;
.quit

Python Query Example

import sqlite3
from datetime import datetime, timedelta

conn = sqlite3.connect('audio_events.db')
cursor = conn.cursor()

# Get events from last hour
one_hour_ago = (datetime.now() - timedelta(hours=1)).isoformat()
cursor.execute('''
    SELECT timestamp, event_type, confidence
    FROM audio_events
    WHERE timestamp > ?
    ORDER BY timestamp DESC
''', (one_hour_ago,))

for row in cursor.fetchall():
    print(f"{row[0]} | {row[1]} | {row[2]:.1%}")

conn.close()

🐛 Troubleshooting

Model Errors

"Index out of bounds"

• Fixed! Audio preprocessing now matches model input shape [1, 44032]

"No module named 'tflite_runtime'"

# On PC: Use full TensorFlow
pip install tensorflow

# On Raspberry Pi: Install tflite_runtime
pip3 install --extra-index-url https://google-coral.github.io/py-repo/ tflite_runtime

Audio Issues

"No default input device"

# List devices
python -c "import sounddevice as sd; print(sd.query_devices())"

# Set specific device in config.py
AUDIO_DEVICE_INDEX = 1  # Replace with your device index

"PortAudio error"

# Windows: Install Microsoft C++ Redistributable
# Raspberry Pi:
sudo apt-get install portaudio19-dev
pip3 install sounddevice --force-reinstall

Configuration Errors

"config.py not found"

• Make sure config.py exists in the same directory
• Never delete or rename config.py

"ImportError: No module named 'config'"

# Make sure you're in the right directory
cd ece-198
python raspberry_pi_monitor.py

🔐 Security Best Practices

Protect API Keys

1. Never commit config.py with real API keys to GitHub

   • Already in .gitignore

2. Use environment variables for production:

import os
EMAIL_PASSWORD = os.environ.get('EMAIL_PASSWORD', 'default-value')

3. Restrict file permissions on Raspberry Pi:

chmod 600 config.py
chmod 600 firebase-credentials.json

Firebase Security Rules

// Firestore rules
rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /audio_events/{document} {
      // Only allow authenticated writes from your device
      allow write: if request.auth != null;
      allow read: if request.auth != null;
    }
  }
}

🚀 Deployment to Raspberry Pi

See RASPBERRY_PI_SETUP.md for complete deployment guide including:

• System setup and dependencies
• File transfer methods
• Auto-start on boot (systemd service)
• Remote access and monitoring
• Performance optimization

📈 Performance Tips

Reduce CPU Usage

# Lower sample rate (less quality, faster)
SAMPLE_RATE = 16000  # Instead of 44100

# Process longer clips less frequently
AUDIO_DURATION = 2.0  # Instead of 1.0

Reduce Database Size

# Higher confidence threshold = fewer logs
CONFIDENCE_THRESHOLD = 0.90  # Instead of 0.80

Disable Verbose Output

VERBOSE = False  # Only show logged events

📝 Development Notes

Model Information

• Source: Teachable Machine (Google)
• Format: TensorFlow Lite
• Input: Raw audio waveform [1, 44032] samples (~1 second at 44.1kHz)
• Output: 5 class probabilities (softmax)
• Classes: 0 Alarms, 1 Background Noise, 2 Clanging, 3 Help, 4 Movement

Testing on PC vs Raspberry Pi

• PC: Use full TensorFlow (larger but has GPU support)
• Pi: Use tflite_runtime (10x smaller, optimized for ARM)
• Same code works on both thanks to try/except import

🤝 Contributing

Issues and pull requests welcome! Areas for improvement:

• Add more notification channels
• Web dashboard for real-time monitoring
• Model retraining pipeline
• Multi-device coordination
• Advanced audio preprocessing

📄 License

MIT License - See LICENSE file for details

🆘 Support

• Issues: https://github.com/KerelosGayed/ece-198/issues
• Documentation: See RASPBERRY_PI_SETUP.md for deployment details

---

Built with ❤️ for ECE-198

🔐 Security Best Practices

1. Never commit master-key.txt to git

   echo "master-key.txt" >> .gitignore

2. Rotate encryption keys periodically

   • Back up old key
   • Generate new key
   • Re-encrypt existing data

3. Use environment variables for secrets

   # Never hardcode credentials in code
   export MONGO_URI="mongodb+srv://..."

4. Restrict MongoDB user permissions

   • Read/write only to required database
   • No admin privileges

5. Enable MongoDB Atlas audit logs

   • Track all data access
   • Alert on suspicious activity

📞 Support

For issues specific to:

• Teachable Machine: Check model.tflite input shape matches preprocessing
• MongoDB: Verify connection string and network access
• Audio: Test with arecord before running Python
• Raspberry Pi: Ensure system updated and USB power adequate

🎓 Next Steps

1. Collect training data in actual hospital environment
2. Retrain model with real samples for better accuracy
3. Create nurse dashboard to visualize MongoDB data
4. Add light sensor for environmental context
5. Implement real-time alerts for critical events