This is the official implementation for CVPR 2025 paper "Open Set Label Shift with Test Time Out-of-Distribution Reference".
Paper link: Main Paper PDF, Supplementary PDF, Youtube Video.
The project is established largely on the OpenOOD 1.5 project publicly available. If you find this repository useful or use this code in your research, please cite the following paper(s):
@inproceedings{ye2025open,
title={Open Set Label Shift with Test Time Out-of-Distribution Reference},
author={Ye, Changkun and Tsuchida, Russell and Petersson, Lars and Barnes, Nick},
booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
pages={30619--30629},
year={2025}
}
as well as the OpenOOD paper:
@article{zhang2023openood,
title={OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection},
author={Zhang, Jingyang and Yang, Jingkang and Wang, Pengyun and Wang, Haoqi and Lin, Yueqian and Zhang, Haoran and Sun, Yiyou and Du, Xuefeng and Li, Yixuan and Liu, Ziwei and Chen, Yiran and Li, Hai},
journal={arXiv preprint arXiv:2306.09301},
year={2023}
}
and other OOD detection papers if you are comparing with.
If you are only interest in the proposed Open Set Label Shift estimation and correction algorithm. The implementation is provided in "OpenSetLabelShift/openood/label_shift/" (https://github.com/ChangkunYe/OpenSetLabelShift/tree/main/openood/label_shift).
Please follow the instruction in README-openood.md for the OpenOOD project requirement, which largely covers the requirement of this project.
Special Requirement: For the label shift estimation models tested, Cvxpy package is used in the closed set label shift BBSE method. Which can be installed via conda:
conda install -c conda-forge cvxpy
Please use bash scripts in "./script/download/" to download the dataset and checkpoints.
After download the checkpoints and datasets, please find and replace across the entire project with:
- Replace "$YOUR_DATASET_PATH" with the actual dataset path you used locally
- Replace "$YOUR_CHECKPOINT_PATH" with the actual checkpoint path you used locally
Please adjust the bash script in "./script/basics/$DATASET/train-**.sh" and train your desired classifier When train the Neural Network classifier from scratch, the recommended hardware setup is as follows:
| Dataset | #GPU | #CPU |
|---|---|---|
| CIFAR10/100 | ≥ 2 Gb | > 4 + 1 threads |
| ImageNet-200 | ≥ 12 Gb | > 16 + 1 threads |
One Nvidia RTX 2080Ti is sufficient for the training. But it's better to have at least 16 Gb GPU space if you want to test ImageNet 1k.
To test existing models' performance under label shift, adjust the dataset path "$data_path" and checkpoint path "$ckpt_path" in the bash script "./test_OSLS.sh" and run:
bash ./test_OSLS.sh
The performance will be printed. Important performance will be recorded in "./results/ood/" as ".csv" files (folder will be created automatically).
Please see LICENSE and LICENSE-openood
For other questions, please raise issues or contact author at changkun.ye@anu.edu.au.
P.S. Please also consider checking issues in the OpenOOD project at https://github.com/Jingkang50/OpenOOD.