🧠 AI-Powered Mental Health & Clinical Decision Support System
An end-to-end AI-driven mental health application designed for hospitals and doctors to analyze patient behavior, generate clinical reports, and evaluate doctor performance using computer vision, speech analysis, and large language models.
🚀 Project Overview
This platform assists mental health professionals by automatically analyzing patient facial expressions, voice patterns, and conversation context during a clinical session and generating AI-assisted reports to support diagnosis and supervision. The system supports role-based access for hospitals and doctors and provides analytics dashboards, EMR automation, and supervision insights.
👥 User Roles & Authentication 🏥 Hospital Admin Secure login Manage and onboard doctors View doctor-level activity and reports
👩⚕️ Doctor Secure login Access personalized dashboard View patient analytics and reports Review AI-generated supervision feedback
📊 Doctor Dashboard After login, doctors can access a personalized analytics dashboard, including: Total patients visited Daily / weekly / monthly visit trends Interactive graphs and statistics Historical session data This helps doctors quickly understand their clinical workload and trends.
📁 Report Management System Centralized report section for each doctor Stores all session-based patient reports Easy access to historical clinical insights Organized by date and patient/session
🧬 EMR Copilot (Core Feature) This is the main pillar of the project The EMR Copilot automates clinical documentation and behavioral analysis during a live doctor-patient session.
🎥 What It Captures Facial expressions via camera (emotion & affect analysis) Voice patterns via microphone (tone, stress, variation) Doctor-patient conversation context throughout the session
⚙️ How It Works Session starts with camera and microphone enabled Patient expressions and voice signals are continuously analyzed Entire session conversation is observed and summarized Collected insights are securely sent to AWS Bedrock A structured AI-generated clinical report is returned
🧾 Output Session summary Emotional and behavioral indicators Communication patternsAI-assisted observations to support diagnosis
🧑🏫 Supervision Report (Doctor Performance Analysis) In addition to patient reports, the system generates a Supervision Report focused on doctor performance.
📈 What It Evaluates Quality of interaction Communication effectiveness Session flow and engagement Overall performance score
🎯 Purpose Self-assessment for doctors Training and improvement insights Clinical supervision support Doctors can view these reports directly from their dashboard.
🏗️ Tech Stack (High-Level) Backend: FastAPI / Python Frontend: Web-based dashboard Computer Vision: Facial expression analysis models Audio Processing: Voice pattern analysis AI / LLM: AWS Bedrock for report generation Database: Relational database for users, reports, sessions
🔐 Security & Privacy Role-based access control Secure authentication No model weights stored in the repository Designed with patient data privacy in mind
📌 Notes Trained ML/DL model weights are not included in this repository due to size constraints. Models can be loaded externally or retrained using provided scripts. This project is intended for educational, research, and clinical support purposes.
🌱 Future Enhancements Real-time emotion visualization Multi-language speech analysis Doctor-to-doctor comparative analytics Integration with hospital EMR systems Deployment with containerization (Docker)
👨💻 Author Jayesh Naidu Machine Learning Engineer | AI & Data Science Enthusiast Focused on AI for Healthcare, Computer Vision, and Applied Machine Learning
⭐ Acknowledgements AWS Bedrock for generative AI capabilities Open-source ML and CV research community