DLMAICLNN01

This project demonstrates continual learning concepts using the Avalanche library for PyTorch. It implements various continual learning strategies on the SplitMNIST benchmark to showcase how models can learn sequentially from different tasks while managing catastrophic forgetting.

Project Overview

This repository contains implementations of continual learning algorithms using the Avalanche framework. The primary focus is on demonstrating how neural networks can learn from sequential tasks without forgetting previously learned information, which is a key challenge in continual learning.

Installation

To run this project, you'll need to install the required dependencies:

pip install -r requirements.txt

The requirements include:

• torch - PyTorch deep learning framework
• avalanche-lib - Avalanche continual learning library
• numpy - Numerical computing library
• matplotlib - Plotting library

Project Structure

• minimal_example_avalance.py - A minimal implementation of continual learning with SplitMNIST
• compute_metrics_avalanche.py - More comprehensive implementation with detailed metrics
• requirements.txt - Project dependencies
• .gitignore - Git ignore file

Usage

Running Examples

To run the minimal example:

python minimal_example_avalance.py

To run the comprehensive example with metrics:

python compute_metrics_avalanche.py

Code Explanation

Both scripts demonstrate the following key components:

1. Benchmark Setup: SplitMNIST dataset split into 5 experiences
2. Model Definition: SimpleMLP neural network
3. Training Strategy: Naive continual learning approach
4. Evaluation: Comprehensive metrics including accuracy, loss, and forgetting

The compute_metrics_avalanche.py file includes more detailed metrics:

• Accuracy metrics (minibatch, epoch, experience, stream)
• Loss metrics (minibatch, epoch, experience, stream)
• Forgetting metrics
• Timing metrics
• CPU usage metrics

Continual Learning Concepts

This project demonstrates fundamental continual learning concepts:

• Experience-based Learning: Tasks are presented sequentially as different experiences
• Catastrophic Forgetting: The challenge of retaining knowledge while learning new tasks
• Sequential Learning: Models learn from data streams without access to previous data
• Evaluation Metrics: Comprehensive metrics to assess performance and forgetting

Key Features

• Implements SplitMNIST continual learning benchmark
• Uses the Avalanche library for robust continual learning framework
• Demonstrates both basic and advanced continual learning approaches
• Includes comprehensive evaluation metrics
• Shows practical implementation of continual learning strategies

Requirements

• Python 3.7+
• PyTorch
• Avalanche library
• NumPy
• Matplotlib

License

This project is created for educational purposes as part of the DLMAICLNN01 course.