ECFOR – End-to-End Communication Framework for Speech and Sign Language Translation

📌 Overview

ECFOR is an end-to-end web-based communication framework designed to bridge the communication gap between hearing/speaking individuals and people with speech or hearing impairments.
The system enables bi-directional translation between:

• Speech → Sign Language
• Sign Language → Text & Speech

By integrating computer vision, deep learning, and speech processing, ECFOR provides an accessible, real-time, and scalable communication solution.

---

🎯 Problem Statement

Individuals with speech or hearing impairments often face significant communication barriers in everyday interactions. Most existing tools are limited to one-directional translation, lack real-time performance, or fail to generalize well across different users and environments.

ECFOR addresses these challenges by offering a unified, real-time, two-way communication platform.

---

🚀 Key Features

• 🔊 Speech to Sign Language translation
• 🤟 Sign Language (video) to text and speech
• 🎥 Real-time video capture and processing
• 🧠 Deep learning–based gesture recognition
• 🌐 Web-based interface for accessibility
• 🔐 User authentication and session management
• 📁 Modular and extensible system design

---

🏗️ System Architecture

The system consists of three core modules:

1️⃣ Speech to Sign Language Module

• Captures speech input via microphone
• Converts speech to text using speech recognition
• Maps recognized text to corresponding sign language gestures
• Uses the WLASL dataset for sign representations

2️⃣ Sign Language to Text & Speech Module

• Captures real-time video input from a camera
• Processes frames using OpenCV
• Recognizes hand gestures using a MobileNet-based deep learning model
• Converts recognized signs into text
• Optionally converts text into synthesized speech

3️⃣ Web Application Layer

• Built using Django
• Manages routing, backend logic, and user authentication
• Integrates ML modules with the frontend
• Handles media files and real-time interaction

---

🧰 Tech Stack

Backend

• Python
• Django

Machine Learning & Computer Vision

• OpenCV
• MobileNet
• WLASL Dataset
• NumPy
• TensorFlow / PyTorch (model dependent)

Speech Processing

• Speech-to-Text APIs
• Text-to-Speech engines

Frontend

• HTML
• CSS
• JavaScript
• Django Templates

---

📂 Project Structure

Ecfor/
├── Authentication/        # User authentication logic
├── Communication/         # Core communication modules
├── Ecfor/                 # Django project configuration
├── speech_to_txt/         # Speech-to-text processing
├── static/                # Static files (CSS, JavaScript)
├── templates/             # HTML templates
├── manage.py              # Django entry point
├── merged_drum.avi        # Sample sign language video
├── output.mp3             # Generated speech output
├── text_to_speech.txt     # Intermediate text output
└── README.md              # Project documentation

---

⚙️ Installation & Setup

Prerequisites

• Python 3.8 or higher
• pip
• Virtual environment (recommended)
• Webcam and microphone (for full functionality)

Installation Steps

# Clone the repository
git clone https://github.com/Hasan-Saju/Ecfor.git
cd Ecfor

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Apply database migrations
python manage.py migrate

# Run the development server
python manage.py runserver

🧪 Dataset

• WLASL (Word-Level American Sign Language Dataset)
• Used for training and evaluating the sign language recognition model
• Supports word-level gesture classification

---

📊 Impact

• Enables real-time communication for individuals with speech or hearing impairments
• Reduces reliance on human interpreters
• Improves accessibility and social inclusion
• Applicable to education, healthcare, and public service domains

---

🙏 Acknowledgements

• WLASL Dataset contributors
• OpenCV and deep learning open-source communities