Despite advancements in technology, millions of deaf and mute individuals around the world still face significant barriers in everyday communication. From casual conversations and accessing emergency help to participating in online meetings and forming social connections, the lack of inclusive communication tools leads to isolation, misunderstanding, and exclusion.
Traditional methods like text messaging or human interpreters are not always accessible, affordable, or real-time. This communication gap not only limits personal and professional opportunities but also impacts emotional well-being and mental health.
There is a pressing need for an intelligent, accessible, and real-time solution that empowers deaf and mute individuals to communicate effortlessly, connect meaningfully, and feel supported—just like anyone else.
SIGNOVA-X addresses this gap by leveraging AI, computer vision, and natural language processing to build a bridge between sign language and spoken language, enabling seamless two-way communication and emotional support in an inclusive digital environment.
Enable seamless two-way communication between deaf/mute users and others. SIGNOVA-X captures live sign language video, extracts frames, and uses advanced AI models to convert it into accurate English text, fostering natural interactions.
A dual-interface screen designed for real-time interactions. One side lets deaf users express through sign language, while the other allows hearing users to respond via voice, which is converted into sign actions by an animated avatar—ensuring fluid conversation.
Using speech-to-text and NLP models, spoken input from hearing users is translated into sign language performed by a dynamic 3D avatar. This brings expressive communication to life, helping bridge emotional and social understanding.
SIGNOVA-X enhances social connectivity using an AI-driven profile-matching algorithm. During sign-up, users input hobbies, interests, and preferences, which are analyzed to recommend compatible friends and expand their social circles.
Address emotional struggles and isolation with a 24/7 motivational chatbot tailored for the deaf community. It offers empathetic support to promote mental resilience and reduce feelings of loneliness.
In urgent situations, users can shake their phone three times to automatically alert emergency contacts and nearby authorities. The alert includes the user's live location, ensuring fast and silent assistance.
Bidirectional communication powered by speech recognition and sign prediction. Supports both sign-to-text and text-to-sign flows for barrier-free, real-time conversations.
User profiles store details like hobbies, favorite activities, and communication styles. These preferences personalize friend suggestions, chatbot tone, and overall app behavior for a tailored user experience.
A clean, intuitive interface built with accessibility in mind. Designed to accommodate motor and visual impairments, ensuring effortless use by all users.
All user interactions—including sign, voice, and location data—are securely processed using end-to-end encryption. SIGNOVA-X follows strict privacy standards to keep your communication confidential.
At SIGNOVA-X, we’re committed to pushing the boundaries of accessibility, inclusion, and meaningful human connection. Below are the upcoming features and innovations we plan to integrate to further empower the deaf and mute community:
Imagine a virtual space where deaf individuals can actively participate in online meetings.
- Our upcoming Google Meet Integration will:
- Detect sign language from the user's webcam in real time.
- Convert it into spoken English audio for other participants.
- This ensures seamless two-way communication in both professional and social virtual settings.
Deaf individuals often miss cues like someone calling their name.
- Future versions of SIGNOVA-X will:
- Use ambient sound detection to recognize when the user’s name is spoken nearby.
- Trigger a gentle vibration alert on their phone.
- This feature helps users stay alert and engaged in their environment.
Many users silently struggle with loneliness, anxiety, or depression due to communication barriers.
- SIGNOVA-X aims to offer:
- In-app access to licensed therapists trained in supporting people with disabilities.
- Mental health resources and empathy-driven care within a safe space.
To foster inclusive communication for everyone:
- We’re developing a sign language learning portal featuring:
- Video lessons
- Quizzes and practice sessions
- Interactive chats with an AI avatar
- This helps both deaf and hearing users become better communicators.
Track and visualize your personal communication journey.
- A dynamic dashboard showing:
- Improvements in fluency, vocabulary, and usage
- Stats based on chatbot and real-time interaction history
- Encourages self-growth and celebrates progress.
Sign languages are not universal. To support global diversity:
- SIGNOVA-X will support:
- Regional sign languages like ISL, ASL, BSL, and more.
- Real-time translation of spoken languages into their local sign language variants.
We envision a supportive community built on empathy.
- SIGNOVA-X will introduce:
- A volunteer listener program connecting hearing individuals with deaf users.
- Opportunities to offer support, companionship, or simply be a friend.
This initiative strengthens community bonds through shared conversations and mutual respect.
SIGNOVA-X is designed to help deaf and mute individuals communicate, connect, and feel supported. Here's a simple step-by-step guide to get started and explore each powerful feature:
Start by creating your account or logging in with your existing credentials.
- Provide your name, email, and password.
- Fill out your interests, hobbies, and communication preferences to personalize your experience.
- Add emergency contact details for SOS functionality.
Navigate to the Live Communication section of the app to experience real-time two-way interaction between deaf/mute and hearing users. This section is powered by advanced AI and gesture recognition technology and now features two smart toggle options:
- 🎥 Upon selecting the "From Sign" button, the device camera opens.
- 🔼 The upper half of the screen displays live video input of the user's hand gestures.
- 🧠 Using AI models powered by MediaPipe & OpenCV, SIGNOVA-X analyzes gestures in real time.
- 🔽 The lower half shows the translated English text corresponding to the recognized signs.
- ✅ Helps users with hearing/speech impairments express themselves naturally through sign gestures.
- 💬 The user can either type a message or speak into the microphone.
- 🧠 Input is processed through speech-to-text (if spoken) or parsed (if typed).
- 🧍♂️ An animated avatar visually replicates the message in sign language.
- 👁️🗨️ Enables deaf/mute users to visually receive messages in a fully accessible format.
- 🔁 Bridges the communication gap in a seamless, inclusive, and intuitive manner.
Go to the Profile section to update or view your personal info.
- Add interests such as music, movies, hobbies, or favorite topics.
- SIGNOVA-X uses this to recommend compatible users, helping you form meaningful friendships.
Feeling down or alone? Navigate to the Chatbot section.
- Engage with our AI-powered chatbot trained to provide:
- Emotional support
- Motivational messages
- Casual conversations tailored to your mood
Your 24/7 digital friend—always available to listen without judgment.
In an emergency, when you can’t speak or type:
- Shake your phone 3 times quickly.
- This sends an SOS alert to your emergency contacts and nearby police with:
- A predefined emergency message
- Your real-time GPS location
Get help instantly with a simple gesture—no words needed.
Visit the Community section to engage with others.
- View AI-suggested friend recommendations.
- Chat with others or join group discussions.
- Share stories, experiences, and feel part of a supportive, inclusive environment.
Stay tuned for exciting upcoming features:
- 📊 Progress Tracker – Track your communication improvements over time.
- 🤟 Learning Section – Practice and learn sign language at your own pace.
- 🗳️ Feedback Portal – Share ideas and suggestions to help shape SIGNOVA-X’s future.
Follow the steps below to set up and run the Flutter frontend for the SIGNOVA-X application.
📝 Prerequisites Before you start, make sure you have installed the following on your system: Flutter SDK VS Code Dart and Flutter extensions for VS Code Android emulator or physical device Python 3.10.x
git clone https://github.com/SIGNOVA-X/frontend.gitcd frontendTo run the project locally, create a .env file in the root directory and add the following environment variables:
You can generate the google Generative AI API Key from Google AI Studio
For Pinata and Nagrok setup look at the backend repo
# 🔑 Google Generative AI API Key
GENERATIVE_AI_APIKEY=""
# 📦 Pinata (IPFS) API Keys
PINATA_API_KEY=""
PINATA_SECRET_API_KEY=""
# 🌐 Ngrok Public URL for Backend Access
NGROK_URL=""flutter pub get Run the Flutter AppSIGNOVA-X is more than just a backend system — it's a step toward inclusive, AI-driven communication. By combining deep learning, computer vision, and generative AI, it empowers the deaf and mute community to connect, express, and engage with the world in real time. Whether it's through sign-to-text conversion, avatar-based animations, or mental health support, SIGNOVA-X ensures that technology serves everyone, equally.
We believe in a future where communication has no barriers — and this is just the beginning.
Feel free to explore, contribute, or get in touch to join us on this mission. 🤝✨
- World Health Organization (WHO) - Causes of Hearing Loss
- Indian Deaf Association Survey 2023 - Social Isolation Among Deaf Individuals