GIFNet [CVPR 2025]

This is the offical implementation for the paper titled "One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion".

Paper & Supplement

"A comparison of the versatility and efficiency of advanced multi-task fusion methods and the proposed pixel-focused low-level interaction paradigm."

Environment

You can setup the required Anaconda environment by running the following prompts:

conda create -n GIFNet python=3.8.17
conda activate GIFNet
pip install -r requirements.txt

Test

The single required checkpoint is avaiable in the folder "model"

Arguments:

"--test_ir_root": Root path for the infrared input.
"--test_vis_root": Root path for the visible input.
"--VIS_IS_RGB": Visible input is stored in the RGB format.
"--IR_IS_RGB": Infrared input is stored in the RGB format.
"--save_path": Root path for the fused image.

Infrared and Visible Image Fusion (IVIF):

python test.py  --test_ir_root "images/IVIF/ir" --test_vis_root "images/IVIF/vis" --save_path "outputsIVIF" --VIS_IS_RGB 

Multi-Focus Image Fusion (MFIF):

python test.py  --test_ir_root "images/MFIF/nf" --test_vis_root "images/MFIF/ff" --save_path "outputsMFIF" --IR_IS_RGB --VIS_IS_RGB

Multi-Exposure Image Fusion (MEIF):

python test.py  --test_ir_root "images/MEIF/oe" --test_vis_root "images/MEIF/ue" --save_path "outputsMEIF" --IR_IS_RGB --VIS_IS_RGB 

Medical Image Fusion:

python test.py  --test_ir_root "images/Medical/pet" --test_vis_root "images/Medical/mri" --save_path "outputsMedical" --IR_IS_RGB

Near-Infrared and Visible Image Fusion (NIR-VIS)

python test.py  --test_ir_root "images/NIR-VIS/nir" --test_vis_root "images/NIR-VIS/vis" --save_path "outputsNIR-VIS" --VIS_IS_RGB

Remote Sensing Image Fusion (Remote)

Step1 : Seprately fuse different bands of the multispectral image with the panchromatic image

(Python)

python test.py  --test_ir_root "images/Remote/MS_band1" --test_vis_root "images/Remote/PAN" --save_path "outputsRemoteBand1"
python test.py  --test_ir_root "images/Remote/MS_band2" --test_vis_root "images/Remote/PAN" --save_path "outputsRemoteBand2"
python test.py  --test_ir_root "images/Remote/MS_band3" --test_vis_root "images/Remote/PAN" --save_path "outputsRemoteBand3"
python test.py  --test_ir_root "images/Remote/MS_band4" --test_vis_root "images/Remote/PAN" --save_path "outputsRemoteBand4"

Step2: Aggregate the separate fused channels together

(Matlab Environment)

Matlab_SeparateChannelsIntoFused

Training

Training Set

• Baidu Drive (code: x2i6)
• Google Drive

Instructions

1. Extract training data from the zip and put them in the "train_data" folder.

2. Run the following prompt to start the training (important parameters can be modified in the "args.py" file):

python train.py --trainDataRoot "./train_data"

The trained models will be saved in the "model" folder (automatically created).

Announcement

• 2025-04-15 The training code is now available.
• 2025-03-11 The test code for all image fusion tasks is now available.
• 2025-02-27 This paper has been accepted by CVPR 2025.

Contact Informaiton

If you have any questions, please contact me at chunyang_cheng@163.com.

(Please clearly note your identity, institution, purpose)

Highlight

• Collaborative Training: Uniquely demonstrates that collaborative training between low-level fusion tasks yields significant performance improvements by leveraging cross-task synergies.
• Bridging the Domain Gap: Introduces a reconstruction task and an augmented RGB-focused joint dataset to improve feature alignment and facilitate effective cross-task collaboration, enhancing model robustness.
• Versatility: Advances versatility over multi-task fusion methods by reducing computational costs and eliminating the need for task-specific adaptation.
• Single-Modality Enhancement: Pioneers the integration of image fusion with single-modality enhancement, broadening the flexibility and adaptability of fusion models.

Citation

If this work is helpful to you, please cite it as:

@InProceedings{Cheng_2025_CVPR,
    author    = {Cheng, Chunyang and Xu, Tianyang and Feng, Zhenhua and Wu, Xiaojun and Tang, Zhangyong and Li, Hui and Zhang, Zeyang and Atito, Sara and Awais, Muhammad and Kittler, Josef},
    title     = {One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {28102-28112}
}