Interpreting Multi-Branch Anti-Spoofing Architectures

This repository contains the implementation of the methodology described in the article "Interpreting Multi-Branch Anti-Spoofing Architectures: Correlating Internal Strategy with Empirical Performance".

The project provides a framework for analyzing the internal decision-making process of the AASIST3 architecture (Audio Anti-Spoofing using Integrated Spectro-Temporal GNNs) applied to the ASVspoof 2019 dataset.

Methodology Overview

The analysis pipeline consists of three main phases:

1. Spectral Feature Extraction:

   • We extract intermediate activation maps from 14 key architectural components (HSGAL layers, Pooling layers, Global Attention modules).
   • Instead of raw activations, we compute the eigenvalue spectrum of the covariance matrices of these activations. This provides a robust, rotation-invariant signature of the internal representation.

2. Meta-Classification:

   • These spectral features are used to train a CatBoost classifier to predict the specific attack type (A07-A19) or Bonafide speech.
   • This "probe" model learns which internal components contain the most discriminatory information for each attack.

3. SHAP-based Strategy Quantification:

   • We employ TreeSHAP to explain the predictions of the meta-classifier.
   • SHAP values are aggregated to quantify the contribution of each branch (B0-B3) and global module (GAT-S/T).
   • The model's behavior is classified into operational archetypes: Effective Specialization, Effective Consensus, Ineffective Consensus, and Flawed Specialization.

Project Structure

aasist3_interp/
├── requirements.txt       # Python dependencies
├── main.py                # Entry point for the entire pipeline
├── src/
│   ├── config.py          # Configuration (Paths, Hyperparameters, Model Args)
│   ├── model/             # AASIST3 Architecture implementation
│   │   ├── aasist3.py     # Main model class
│   │   ├── kan.py         # KANLinear layers
│   │   ├── gat.py         # Global Attention layers
│   │   ├── hs_gal.py      # HSGAL layers
│   │   └── ...
│   ├── data/
│   │   └── loader.py      # Data loading and preprocessing (ASVspoof)
│   └── analysis/
│       ├── extractor.py   # Hook-based feature extraction (Eigenvalues)
│       ├── interpret.py   # CatBoost training and SHAP calculation
│       └── plotting.py    # Visualization (Violin plots, Decision plots)

Prerequisites

• Python 3.8+
• NVIDIA GPU (Recommended for feature extraction speed)
• ASVspoof 2019 Logical Access (LA) Dataset

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/aasist3_interp.git
   cd aasist3_interp

2. Install dependencies:

   pip install -r requirements.txt

Dataset Setup

1. Download the ASVspoof 2019 Logical Access dataset.

2. Update the path in main.py (or modify src/config.py) to point to your dataset location:

   # In main.py
   base_path = '/path/to/ASVspoof2019/LA'

   The directory structure is expected to match the standard ASVspoof layout (ASVspoof2019_LA_cm_protocols, ASVspoof2019_LA_eval/flac).

Usage

Run the full analysis pipeline:

python main.py

What happens during execution:

1. Model Loading: Downloads pre-trained AASIST3 weights from HuggingFace (MTUCI/AASIST3) if not present.
2. Feature Extraction: Processes the evaluation set, hooks into specified layers, computes eigenvalues, and constructs a tabular dataset (X, y).
3. Meta-Training: Trains a CatBoost classifier on the spectral features.
4. Interpretation: Calculates SHAP values for the test set.
5. Visualization: Generates analysis plots in the output_results/ directory.

Outputs

The script generates the following artifacts in output_results/:

• detailed_stats.csv: Raw statistical metrics of SHAP values per component.
• summary_stats.csv: Aggregated contribution shares for major branches.
• violin_{attack_type}.png: Diverging violin plots showing the contribution distribution (positive for spoof, negative for bonafide) and the specialization score.
• decision_plot_{attack_type}.png: SHAP decision plots visualizing the cumulative evidence accumulation across branches.

References

• Model Implementation: Based on the official AASIST3 architecture using Kolmogorov-Arnold Networks (KAN).
• Methodology: "Interpreting Multi-Branch Anti-Spoofing Architectures: Correlating Internal Strategy with Empirical Performance".