MIMIR: Masked Image Modeling for Mutual Information Based Adversarial Robustness

Environment settings

check requirements.txt

Install AutoAttack

pip install git+https://github.com/fra31/auto-attack

Evaluation and Training Commands

check ./script

Checkpoints

MIMIR + 3 step PGD on swin models (implemented using ARES, see pgd_AT_ares)

Model	Natural	AA	CheckPoint
Swin-base	76.62	55.90	Link
Swin-large	78.62	59.68	Link

MIMIR + more advanced fine-tuning (2-step APGD adversarial training, 300 epochs) for SOTA performance on ImageNet-1K

eps 4

Model	Natural	AA	CheckPoint
ViT-S	71.00	46.10	Link
ViT-B	76.32	54.28	Link

CIFAR-10

eps 8

Model	Natural	AA	CheckPoint
ViT-T	84.82	52.96	Link
ViT-S	88.11	53.18	Link
ViT-S (5m edmdata, 100 ft epoch)	91.94	61.06	Link
ViT-B	89.30	54.55	Link
ViT-B (5m edmdata, 100 ft epoch)	92.82	63.43	Link
ConViT-T	80.74	45.04	Link
ConViT-S	87.49	52.54	Link
ConViT-B	89.30	55.64	Link

ImageNet-1K

eps 2

Model	Natural	PGD 20	CheckPoint
ViT-S	74.60	54.56	Link
ViT-B	75.88	55.42	Link
CaiT-XXS24	73.39	53.39	Link
CaiT-S36	76.05	56.78	Link

eps 4

Model	Natural	PGD 20	CheckPoint
ViT-S	71.29	40.98	Link
ViT-B	73.22	41.26	Link
CaiT-XXS24	69.90	40.53	Link
CaiT-S36	73.57	40.03	Link

MIMIR Pre-train checkpoints

Dataset	Model	CheckPoint
CIFAR-10	ViT-T	Link
CIFAR-10	ViT-S	Link
CIFAR-10	ViT-B	Link
CIFAR-10	ConViT-T	Link
CIFAR-10	ConViT-S	Link
CIFAR-10	ConViT-B	Link
ImageNet-1K	ViT-S	Link
ImageNet-1K	ViT-B	Link
ImageNet-1K	CaiT-XXS24	Link
ImageNet-1K	CaiT-S36	Link
ImageNet-1K	Swin-base	Link
ImageNet-1K	Swin-large	Link

Acknowlegements

This repository is built upon the following repositories:

https://github.com/facebookresearch/mae

https://github.com/wzekai99/DM-Improves-AT

https://github.com/yuxi120407/DIB

https://github.com/choasma/HSIC-bottleneck

https://github.com/thu-ml/ares