Skip to content

README.md

Latest commit

 

History

History
173 lines (118 loc) · 5.71 KB

README.md

File metadata and controls

173 lines (118 loc) · 5.71 KB

Sentiment Classification Tradeoffs with RNN, LSTM, and BiLSTM

Author: Dharmitha Niteesha Vattikonda

Overview

This project investigates how different recurrent neural network architectures perform on a real-world sentiment classification task using the IMDB movie reviews dataset. The goal is not only to compare RNN, LSTM, and Bidirectional LSTM models, but to understand how architectural choices, sequence length, optimization strategy, and gradient stability affect performance, training time, and reliability.

Through a controlled experimental setup, the project evaluates tradeoffs between model complexity, convergence behavior, and predictive quality, with an emphasis on selecting configurations that balance accuracy, stability, and computational cost.

Problem Statement

Sentiment classification on long-form text presents challenges related to sequence modeling, gradient stability, and computational efficiency. While simple RNNs struggle with long-range dependencies, more complex architectures such as LSTMs and Bidirectional LSTMs introduce additional training cost and complexity.

Choosing an appropriate architecture and training configuration requires understanding the tradeoffs between accuracy, stability, and runtime. This project addresses that problem by systematically evaluating recurrent model variants under different training conditions to identify practical, high-performing configurations.

Setup Instructions

Python Environment

This project was tested with:


Python 3.10+
TensorFlow 2.12+
scikit-learn 1.5+
matplotlib 3.9+
seaborn 0.13+
pandas 2.2+
numpy 1.26+

You can install dependencies manually or using pip:

pip install -r requirements.txt

Project Structure

sentiment_rnn_project/
│
├── data/
│   └── IMDB Dataset.csv
│
├── results/
│   ├── metrics.csv
│   ├── metrics_seq.csv
│   ├── metrics_opt_act.csv
│   ├── plots/
│   │   ├── architecture_accuracy.png
│   │   ├── seq_accuracy.png
│   │   ├── opt_activation_f1.png
│   │   └── ...
│   └── summaries/
│
├── src/
│   ├── preprocess.py
│   ├── models.py
│   ├── train.py
│   ├── evaluate.py
│   └── utils.py
│
└── README.md

Running the Project

1.Train all models and save results

Run from inside the src/ directory:

python train.py

This will:

  • Train and evaluate all RNN, LSTM, BiLSTM models

  • Run experiments for sequence length and optimizers

  • Save:

    • Trained model (best_lstm_model.h5)
    • Predictions (preds.npy)
    • Evaluation data (yte.npy, lstm_history.npy)
    • CSV logs in /results/

2. Generate Evaluation Plots

After training completes, run:

python evaluate.py

This script:

  • Reads your result CSVs
  • Generates bar, line, scatter, and heatmap plots
  • Saves all images inside /results/plots/

3. Summarize Best Configurations (optional)

To print and save best-performing configurations:

python utils.py

This produces text summaries (e.g. summary_metrics.txt) showing top configurations for each experiment.

Expected Runtime

Model Dataset Size Avg Epoch Time Total Runtime
RNN 50 seq ~40 sec/epoch ~4 min
LSTM 50 seq ~80 sec/epoch ~8–10 min
BiLSTM 50 seq ~130 sec/epoch ~13-15 min
Full experiment suite — — ~30–40 min total on CPU

Using a GPU (e.g., on Google Colab or Kaggle) can reduce runtime by 3–4×.

Output Files

File Description
results/metrics.csv Accuracy/F1/time for RNN, LSTM, BiLSTM
results/metrics_seq.csv Performance across sequence lengths
results/metrics_opt_act.csv Comparison of optimizers & activations
results/plots/ Visualizations (accuracy, F1, learning curves, etc.)
results/best_lstm_model.h5 Saved trained model
results/lstm_history.npy Learning curve data
results/preds.npy / results/yte.npy Predictions and true labels
results/summary_*.txt Text summaries of best configurations

Insights & Tradeoffs

  • LSTM with tanh activation and Adam optimizer provides the best balance of accuracy and training stability.
  • Bidirectional LSTMs increase training time significantly while offering marginal accuracy improvements on this dataset.
  • Adam and RMSProp converge faster and more reliably than SGD, especially for longer sequences.
  • Gradient clipping improves training stability but does not materially improve final accuracy.

Production Considerations

Based on these results, a standard LSTM with moderate sequence length would be preferred in a production setting due to its balance of performance and computational cost. More complex architectures such as BiLSTMs may be justified only when marginal accuracy gains outweigh increased latency and resource usage.

Further improvements could include pretrained embeddings, transformer-based architectures, or model distillation for deployment efficiency.