Head is a modular middleware designed to make any Large Language Model (LLM) emotionally aware without retraining the LLM itself.
It acts as a plug-and-play emotion extraction component that processes user input, detects emotional states, and injects this metadata into the chatbot prompt pipeline.
This repository contains:
- 🧠 Head Module — A BERT/RoBERTa-based multi-label emotion classifier trained on GoEmotions (27 emotions).
- 💬 Emotion-Aware Chat Application — A Streamlit UI that integrates the Head module with an LLaMA-based chatbot using Ollama for real-time empathetic conversation.
- sst2/ — Fine-tuned sentiment model + hyperparameter optimization (source task).
- goemotions/ — Final fine-tuned multi-label emotion model (target task).
✔ Multi-label Emotion Detection (GoEmotions: 27 classes)
✔ Transfer Learning (SST-2 → GoEmotions)
✔ Hyperparameter tuning with Optuna (Bayesian search)
✔ Cleaned Streamlit chatbot with memory
✔ Emotion-conditioned prompting for LLaMA (Ollama)
✔ Fully modular — LLM and emotion model are independent
✔ Ready-to-deploy structure
- GoEmotion (Traning Script to Fine Tune RoBERTa on GoEmotion Dataset)
- SST-2 (Hyper Parameter Tuning Script For BERT Family)
User Input ↓ Emotion Classifier (Head Module - RoBERTa) ↓ Detected Emotions + Confidence Scores ↓ Prompt Builder (adds emotional metadata) ↓ LLaMA (via Ollama) ↓ Empathetic Response ↓ Streamlit UI
| Dataset | Purpose | Format |
|---|---|---|
| SST-2 | Transfer learning source task | Binary sentiment |
| GoEmotions | Final fine-tuning | 27 emotions |
- Base architecture: RoBERTa-base
- Loss: BCEWithLogitsLoss (multi-label)
- Tokenization:
max_length=128 - Label encoding: MultiLabelBinarizer
- Mixed precision training enabled
- Fine-tune RoBERTa on SST-2
- Hyperparameter optimization using Optuna
- Load best checkpoint
- Re-initialize final classifier for 27 emotion labels
- Fine-tune on GoEmotions
- Tune threshold (0.1–0.9 grid) for best macro F1
| Metric | Value |
|---|---|
| Accuracy | ~94% |
| Best LR | 2.28e-5 |
| Batch Size | 16 |
| Epochs | 4 |
| Model | Macro F1 | Micro F1 |
|---|---|---|
| Baseline (no transfer) | ~0.42 | ~0.60 |
| Transfer (SST-2 → GoEmotions) | ~0.45 | ~0.62 |
✨ Transfer learning gave a measurable improvement.
The chatbot uses detected emotions to modulate:
- Tone (warm, supportive, calm)
- Word choice
- Length
- Follow-up questions
Example emotion metadata:
Detected emotions: sadness (0.82), confusion (0.41)
Prompt sent to LLaMA:
You are an empathetic assistant. Use the user's detected emotions to adjust tone. Conversation history: ... Respond to the latest user message.
REPO LINK : ChatBot WebApp
conda create -n env_name conda activate env_name
pip install -r requirements.txt
Minimal requirements:
- transformers
- torch
- scikit-learn
- streamlit
- datasets
- optuna
- langchain-ollama
- requests
(Update paths inside scripts as needed.)
- Add conversation-level emotion tracking
- Use cross-encoders for context-aware emotion detection
- Add speech (audio emotion,video emotion) → multimodal Head
- Compression/distillation for mobile deployment
Pull requests are welcome!
Open an issue if you want to discuss new features.
MIT License (modify as needed)