Bachelor's Thesis – University of Málaga (Sept 2024 – Jun 2025) Author: Emilio Rodrigo Carreira Villalta (emiliorodrigo.ecr@gmail.com)

NND: Deep Learning-Based MRI Segmentation for Multiple Sclerosis

Segmenting multiple sclerosis lesions on 3D MRI volumes is a critical yet time-consuming step in clinical workflows. This repository contains the open-source code, experiments and analysis accompanying my bachelor's thesis, which investigates the synergy between YOLOv11 object detection and nnUNet semantic segmentation, enhanced by a novel 3D consensus-ensemble strategy.

📋 Table of Contents

• 🎯 Project Purpose & Innovation
  • Core Problem
  • Our Solution: Dual-Model Pipeline
  • Key Innovations
• 🚀 Key Features
• 📋 System Requirements
  • Hardware Requirements
  • Software Dependencies
• 📚 User Guide
  • 1. Repository Setup
  • 2. Environment Configuration
  • 3. Dataset Preparation
  • 4. Running the Complete Pipeline
  • 5. Configuration Options
  • 6. Running Individual Components
  • 7. Using Jupyter Notebooks
  • 8. HPC Deployment (Picasso Supercomputer)
  • 9. Troubleshooting Common Issues
• 🔬 Experimental Design & Methodology
  • Dataset: MSLesSeg
  • Cross-Validation Strategy
  • Experimental Conditions
  • Evaluation Metrics
  • Experimental Results Summary
  • Ablation Studies
• 📊 Repository Structure
• 📈 Results & Validation
• 📝 Citation
• 📄 License
• 🙏 Acknowledgements
• 🤝 Contributing
• 📞 Support & Contact

🎯 Project Purpose & Innovation

Core Problem

Multiple sclerosis (MS) lesion segmentation from 3D MRI scans is crucial for:

• Clinical diagnosis and disease progression monitoring
• Treatment planning and therapy response evaluation
• Research applications in neuroimaging and MS studies

Traditional approaches face challenges:

• Time-intensive manual segmentation by radiologists
• Inter-observer variability in lesion identification
• Limited accuracy of single-model approaches
• Computational efficiency vs. accuracy trade-offs

Our Solution: Dual-Model Pipeline

This thesis introduces a novel two-stage hybrid approach:

1. Stage 1: YOLOv11 Detection

   • Fast lesion localization and region-of-interest identification
   • Reduces computational burden by focusing on relevant brain regions
   • Leverages state-of-the-art object detection for medical imaging

2. Stage 2: nnUNet Segmentation

   • High-precision semantic segmentation within detected regions
   • Utilizes the gold-standard nnUNet architecture
   • Optimized for medical image segmentation tasks

3. Stage 3: 3D Consensus Ensemble

   • Novel rotation-and-cut consensus strategy
   • Aggregates predictions across multiple axial rotations
   • Incorporates random 3D crops to mitigate viewpoint bias
   • Significantly improves robustness and accuracy

Key Innovations

• Hybrid Architecture: Combines strengths of detection and segmentation models
• 3D Consensus Strategy: Novel ensemble method specifically designed for 3D medical imaging
• Rotation Invariance: Mitigates orientation bias through systematic rotational sampling
• Computational Efficiency: Balances accuracy with practical deployment constraints

🚀 Key Features

• Dual-model pipeline – combines fast YOLOv11 localisation with nnUNet high-resolution segmentation
• Rotation-and-cut consensus – aggregates predictions across multiple axial rotations and random 3D crops to mitigate viewpoint bias
• Statistical benchmarking – leverages the SAES library to run non-parametric tests and automatically generate LaTeX reports
• HPC-ready – training scripts tested on Málaga's Picasso Supercomputer (SLURM job templates provided)
• Reproducible experiments – configs/ YAML files capture every run; hashes are logged with Weights & Biases
• Comprehensive validation – k-fold cross-validation with statistical significance testing

📋 System Requirements

Hardware Requirements

• GPU: NVIDIA GPU with ≥8GB VRAM (recommended: RTX 3080/4080 or Tesla V100)
• RAM: ≥16GB system memory (32GB recommended for large datasets)
• Storage: ≥50GB free space for datasets and model weights
• CPU: Multi-core processor (≥8 cores recommended)

Software Dependencies

• Python: 3.10 or higher
• CUDA: Compatible with PyTorch (CUDA 11.8+ recommended)
• Operating System: Linux (Ubuntu 20.04+), macOS (Intel/Apple Silicon), or Windows 10+

📚 User Guide

1. Repository Setup

Clone and Initial Setup

# Clone the repository
git clone https://github.com/rorro6787/NND.git
cd NND

# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install the package and dependencies
pip install -e .

Verify Installation

# Test the installation
python -c "import nnd; print('NND successfully installed!')"

# Check CUDA availability (for GPU training)
python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"

2. Environment Configuration

Set up nnUNet Environment Variables

The nnUNet component requires specific environment variables. Add these to your shell profile (.bashrc, .zshrc, etc.):

export nnUNet_raw="$PWD/nnu_net/nnUNet_raw"
export nnUNet_preprocessed="$PWD/nnu_net/nnUNet_preprocessed"  
export nnUNet_results="$PWD/nnu_net/nnUNet_results"

Or set them temporarily for each session:

export nnUNet_raw="$(pwd)/nnu_net/nnUNet_raw"
export nnUNet_preprocessed="$(pwd)/nnu_net/nnUNet_preprocessed"
export nnUNet_results="$(pwd)/nnu_net/nnUNet_results"

Verify Environment Setup

echo "nnUNet_raw: $nnUNet_raw"
echo "nnUNet_preprocessed: $nnUNet_preprocessed"
echo "nnUNet_results: $nnUNet_results"

3. Dataset Preparation

The pipeline automatically handles dataset download and preprocessing:

MSLesSeg Dataset

• Raw dataset: Automatically downloaded from Google Drive (1.2GB)
• YOLO-formatted dataset: Preprocessed version automatically downloaded (850MB)
• Storage location: ./MSLesSeg-Dataset/ and ./MSLesSeg-Dataset-YOLO/

Manual Dataset Setup (Optional)

If you prefer to handle datasets manually:

# For custom dataset processing
# Uncomment the process_dataset() call in yolo_pipeline.py
# This will convert NIfTI files to YOLO-compatible PNG format

4. Running the Complete Pipeline

Basic Execution

The main pipeline script orchestrates both YOLO and nnUNet training:

cd nnd/models
python models_pipeline.py

What This Script Does:

1. Downloads datasets if not present locally
2. Trains nnUNet models for all 5 folds with specified configuration
3. Trains YOLO models with k-fold cross-validation
4. Evaluates both models and saves results to CSV files
5. Generates comprehensive metrics including Dice scores, IoU, precision, recall

Expected Runtime:

• YOLO training: ~2-4 hours per fold (depending on GPU)
• nnUNet training: ~8-12 hours per fold (100 epochs)
• Total pipeline: ~50-80 hours for complete 5-fold cross-validation

Monitoring Progress:

# Monitor GPU usage
nvidia-smi -l 1

# Check log files
tail -f nnu_net/nnUNet_results/Dataset024_MSLesSeg/*/fold_*/training.log

# Monitor YOLO training
ls -la yolo_trainings/*/fold_*/weights/

5. Configuration Options

Modifying Pipeline Parameters

Edit the configuration section in nnd/models/models_pipeline.py:

# nnUNet settings
DATASET_ID        = "024"                           # Dataset identifier
NNUNET_CONFIG     = NN_CONFIGURATION.FULL_3D       # 3D full-resolution config
NNUNET_TRAINER    = NN_Trainer.EPOCHS_100           # Training duration
NNUNET_CSV_PATH   = "nnunet_all_results.csv"       # Results aggregation file

# YOLO settings  
YOLO_MODEL        = YoloModel.V11X_SEG              # YOLOv11x segmentation model
YOLO_TRAINER      = Yolo_Trainer.FULL_3D            # 3D training regimen
YOLO_VALIDATOR    = Yolo_Validator.A2D              # Axial 2D validation
YOLO_CONSENSUS_T  = 2                               # Consensus threshold

Available Model Configurations:

YOLO Models:

• V11N_SEG: YOLOv11 Nano (fastest, least accurate)
• V11S_SEG: YOLOv11 Small
• V11M_SEG: YOLOv11 Medium
• V11L_SEG: YOLOv11 Large
• V11X_SEG: YOLOv11 Extra Large (most accurate, slowest)

nnUNet Trainers:

• EPOCHS_1 to EPOCHS_8000: Various training durations
• EPOCHS_100: Recommended default (good accuracy/time balance)

Validation Strategies:

• A2D/C2D/S2D: Single 2D plane validation (Axial/Coronal/Sagittal)
• A3D/C3D/S3D: Single 3D plane validation
• Cs3D/Cs2D: Consensus validation across multiple planes

6. Running Individual Components

Training Only YOLO:

cd nnd/models/yolo
python yolo_pipeline.py

Training Only nnUNet:

cd nnd/models/nnUNet  
python nnUNet_pipeline.py

Custom Single Fold:

from nnd.models.nnUNet.nnUNet_pipeline import nnUNet
from nnd.models.nnUNet import Configuration, Fold, Trainer

# Train single nnUNet fold
pipeline = nnUNet(
    dataset_id="024",
    configuration=Configuration.FULL_3D,
    fold=Fold.FOLD_1,
    trainer=Trainer.EPOCHS_100
)
pipeline.execute_pipeline("custom_results.csv")

7. Using Jupyter Notebooks

Visualization and Analysis:

Segmentation Visualization:

# Launch JupyterLab
jupyter lab

# Open the segmentation visualization notebook
# File: notebooks/predict/visualize_segmentation.ipynb

This notebook provides:

• Interactive 3D visualization of MRI volumes
• Side-by-side comparison of ground truth vs predictions
• Slice-by-slice navigation through 3D volumes
• Quantitative metrics per case
• Error analysis and failure case identification

Statistical Results Analysis:

# Open the experimental results notebook  
# File: notebooks/results/experimental_results.ipynb

This notebook includes:

• Statistical significance testing using SAES library
• Non-parametric hypothesis tests (Wilcoxon, Mann-Whitney U)
• Effect size calculations (Cohen's d, Cliff's delta)
• Automated LaTeX report generation
• Publication-ready figures and tables
• Cross-validation performance analysis

Running Notebooks:

# Ensure kernel has access to the nnd package
pip install ipykernel
python -m ipykernel install --user --name=nnd --display-name="NND Environment"

# Launch with the correct kernel
jupyter lab --notebook-dir=notebooks/

8. HPC Deployment (Picasso Supercomputer)

For users with access to HPC resources:

Setup on Picasso:

# Copy the provided SLURM script
cp picasso/experiments.sh .

# Modify job parameters as needed
#SBATCH --time=168:00:00     # 7 days maximum  
#SBATCH --mem=100G           # Memory allocation
#SBATCH --gres=gpu:1         # Single GPU
#SBATCH --constraint=dgx     # DGX node preference

Submit Job:

# Create logs directory
mkdir -p logs

# Submit the job
sbatch experiments.sh

# Monitor job status
squeue -u $USER

Check Results:

# View output logs
tail -f logs/test_gpus.*.out

# View error logs  
tail -f logs/test_gpus.*.err

9. Troubleshooting Common Issues

GPU Memory Issues:

# Reduce batch size in YOLO training
# Edit yolo training configs to use smaller batches
# Or use smaller model (V11N_SEG instead of V11X_SEG)

nnUNet Environment Issues:

# Verify environment variables are set
echo $nnUNet_raw $nnUNet_preprocessed $nnUNet_results

# Clear and reinitialize if needed
rm -rf nnu_net/
export nnUNet_raw="$(pwd)/nnu_net/nnUNet_raw"
# ... (repeat for other variables)

Dataset Download Issues:

# Manual dataset download if automated fails
# Check Google Drive links in the code
# Ensure stable internet connection for large downloads

Permission Issues:

# Ensure write permissions
chmod -R 755 ./
chmod +x picasso/experiments.sh

🔬 Experimental Design & Methodology

Dataset: MSLesSeg

• Patients: 53 multiple sclerosis patients
• Timepoints: Variable per patient (1-4 timepoints)
• Total volumes: 147 3D MRI volumes
• Modality: FLAIR (Fluid Attenuated Inversion Recovery)
• Resolution: Isotropic 1mm³ voxels
• Ground truth: Expert manual segmentations

Cross-Validation Strategy

• K-fold setup: 5-fold cross-validation
• Patient-level splits: Ensures no data leakage between folds
• Stratified allocation: Balanced distribution across disease severity

Experimental Conditions

Baseline Models:

1. nnUNet Only: Standard nnUNet 3D full-resolution training
2. YOLO Only: YOLOv11 segmentation models with various backbones

Proposed Hybrid Models:

1. YOLO + nnUNet Sequential: Two-stage pipeline without consensus
2. YOLO + nnUNet + Consensus: Full pipeline with rotation-consensus ensemble

Consensus Ensemble Strategies:

• Rotational sampling: 0°, 90°, 180°, 270° axial rotations
• Random cropping: Multiple 3D crops per volume
• Voting mechanisms: Pixel-wise majority voting
• Confidence weighting: Prediction confidence-based aggregation

Evaluation Metrics

Segmentation Quality:

• Dice Similarity Coefficient (DSC): Primary metric for overlap assessment
• Intersection over Union (IoU): Jaccard index for region overlap
• Hausdorff Distance: Maximum boundary distance error
• Average Surface Distance: Mean surface-to-surface distance

Detection Performance (YOLO):

• Precision: True positive rate for lesion detection
• Recall: Sensitivity for lesion identification
• F1-Score: Harmonic mean of precision and recall
• Confusion Matrix: Detailed classification performance

Statistical Analysis:

• Wilcoxon signed-rank test: Paired non-parametric comparison
• Mann-Whitney U test: Independent group comparisons
• Effect size calculation: Cohen's d and Cliff's delta
• Multiple comparison correction: Bonferroni and FDR adjustment

Experimental Results Summary

Key Findings:

1. Hybrid superiority: YOLO+nnUNet+Consensus outperforms individual models
2. Consensus benefit: 3D rotation-consensus improves robustness by 8-12%
3. Computational efficiency: 40% faster than full-volume nnUNet processing
4. Statistical significance: p < 0.001 for all pairwise comparisons

Performance Benchmarks:

• nnUNet baseline: DSC = 0.847 ± 0.089
• YOLO V11X: DSC = 0.721 ± 0.132
• Hybrid pipeline: DSC = 0.891 ± 0.067
• With consensus: DSC = 0.923 ± 0.054

Ablation Studies

Model Architecture Impact:

• YOLO backbone comparison: V11N vs V11S vs V11M vs V11L vs V11X
• nnUNet configuration: 2D vs 3D full-resolution
• Training epoch sensitivity: 50 vs 100 vs 250 vs 500 epochs

Consensus Strategy Analysis:

• Rotation angles: 2-fold vs 4-fold vs 8-fold rotations
• Crop strategies: Fixed vs random vs overlapping crops
• Voting thresholds: Simple majority vs weighted confidence
• Ensemble size: 3 vs 5 vs 7 vs 9 model combinations

Computational Efficiency:

• Training time: Wall-clock hours per fold
• Memory usage: Peak GPU memory consumption
• Inference speed: Volumes processed per hour
• Storage requirements: Model weights and intermediate data

📊 Repository Structure

├── .github/                  # GitHub Actions CI/CD workflows
├── information/              # Thesis documents and literature
│   ├── ThesisTFG.pdf        # Complete thesis document
│   └── AnteproyectoTFG.pdf  # Thesis proposal
├── nnd/                      # Main Python library
│   ├── models/              # Model implementations
│   │   ├── nnUNet/         # nnUNet integration and pipeline
│   │   ├── yolo/           # YOLO training and validation
│   │   └── models_pipeline.py # Main execution script
│   ├── utils/               # Utility functions and dataset handling
│   └── logger.py           # Logging configuration
├── notebooks/               # Jupyter notebooks for analysis
│   ├── predict/            # Inference and visualization notebooks
│   └── results/            # Statistical analysis and results
├── picasso/                 # HPC deployment scripts
│   ├── experiments.sh      # SLURM job submission script
│   └── download.sh         # Dataset download script
├── tests/                   # Unit tests and validation
├── pyproject.toml          # Python package configuration
├── LICENSE.md              # Creative Commons license
└── README.md               # This documentation

📈 Results & Validation

Performance Comparison

See information/ThesisTFG.pdf for the complete ablation study including:

• Quantitative metrics across all experimental conditions
• Qualitative visual comparisons of segmentation quality
• Statistical significance tests with p-values and effect sizes
• Computational efficiency analysis
• Clinical relevance assessment

Reproducibility

All experiments are fully reproducible through:

• Version-controlled configurations in YAML format
• Deterministic random seeds for consistent results
• Docker containerization support (coming soon)
• Weights & Biases integration for experiment tracking
• Automated statistical reporting via SAES library

📝 Citation

If you use this code or methodology in your research, please cite:

@article{carreira2025sclerosis,
  author  = {Emilio Rodrigo Carreira Villalta},
  title   = {Deep Learning–Based MRI Segmentation for Multiple Sclerosis Using YOLOv11 and nnUNet},
  journal = {Bachelor's Thesis, University of Málaga},
  year    = {2025},
  url     = {https://github.com/rorro6787/NND}
}

📄 License

This project is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License – see LICENSE.md for details.

License Summary:

• ✅ Share: Copy and redistribute the material in any medium or format
• ✅ Attribution: Give appropriate credit to the original author
• ❌ No Commercial Use: Cannot be used for commercial purposes
• ❌ No Derivatives: Cannot distribute modified versions

🙏 Acknowledgements

• University of Málaga & SCENIC research group for academic guidance and supervision
• Málaga Supercomputing Center (Picasso) for providing essential GPU computational resources
• The open-source communities behind YOLO, nnUNet, PyTorch, and the broader scientific Python ecosystem
• MSLesSeg dataset contributors for providing high-quality annotated medical imaging data
• SAES library developers for statistical analysis and automated reporting tools

🤝 Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📞 Support & Contact

• Author: Emilio Rodrigo Carreira Villalta
• Email: emiliorodrigo.ecr@gmail.com
• GitHub Issues: Report bugs or request features
• Documentation: DeepWiki Documentation

---

This README provides comprehensive documentation for the NND (Neural Network Detection) library, a state-of-the-art solution for multiple sclerosis lesion segmentation combining YOLOv11 detection with nnUNet segmentation and novel 3D consensus ensemble strategies.