Towards Robust Influence Functions with Flat Validation Minima

Official PyTorch implementation of the ICML 2025 paper: Towards Robust Influence Functions with Flat Validation Minima

Xichen Ye, Yifan Wu, Weizhong Zhang, Cheng Jin, and Yifan Chen

https://www.arxiv.org/abs/2505.19097

---

Roadmap

We are releasing the code in phases:

• Noisy label experiments (CIFAR-10N / CIFAR-100N)
• Generation tasks: coming soon

Requirements

python>=3.10
pytorch>=2.4
tensorboard
scikit-learn
tqdm
matplotlib
seaborn

Quick start

Run FVM with ResNet-34 on noisy label detection (CIFAR-10N / Aggregate, seed=1):

Step 1: Train the model to obtain training minima

python -m noisy_labels.run_train \
    --model resnet34 \
    --dataset cifar10n \
    --root Datasets/cifar10 \
    --noise_type aggre \
    --noise_file_path noisy_labels/dataset/CIFAR-10_human.pt \
    --batch_size 128 \
    --num_classes 10 \
    --total_epoch 100 \
    --optimizer sgd \
    --learning_rate 0.1 \
    --momentum 0.9 \
    --weight_decay 0.0005 \
    --scheduler multisteplr \
    --milestones 50 \
    --gamma 0.1 \
    --seed 1 \
    --gpu 0 \
    --eval_freq 5 \
    --log_freq 100 \
    --save_model \
    --save_model_freq 10 \
    --save_dir results

Step 2: Tune with SAM to obtain validation minima

python -m noisy_labels.run_tune_sam \
    --model resnet34 \
    --path_checkpoint results/noisy_labels/run_train/cifar100n_noisy/resnet34_s1_<EXP_ID>/ep-100.pth 
    --dataset cifar10n \
    --root Datasets/cifar10 \
    --noise_type aggre \
    --noise_file_path noisy_labels/dataset/CIFAR-10_human.pt \
    --batch_size 128 \
    --num_classes 10 \
    --total_steps 1000 \
    --learning_rate 0.01 \
    --momentum 0.9 \
    --weight_decay 0.0005 \
    --scheduler cosine \
    --T_max 1000 \
    --eta_min 0 \
    --seed 1 \
    --gpu 0 \
    --save_model \
    --save_model_freq 1000 \
    --save_dir results

Step 3: Perform noisy label detection with FVM

python -m noisy_labels.run_mislabel_detection \
    --model resnet34 \
    --path_checkpoint results/noisy_labels/run_tune_sam/cifar10n_aggre/resnet34_s1_<EXP_ID>/step-1000.pth \
    --dataset cifar10n \
    --root Datasets/cifar10 \
    --noise_type aggre \
    --noise_file_path noisy_labels/dataset/CIFAR-10_human.pt \
    --batch_size 100 \
    --num_classes 10 \
    --influence valminima \
    --gpu 1 \
    --save_dir results

Note: <EXP_ID> is an automatically generated folder name that contains
timestamp and hash (e.g., 20250110-214029_VZM1).
Please replace it with the actual folder name produced during your training run.

Scripts

We provide extensive example scripts under the scripts/ directory.