ControlFusion: A Controllable Image Fusion Framework with Language-Vision Degradation Prompts [NeurIPS 2025]

Linfeng Tang^1*,  Yeda Wang^1*,  Zhanchuan Cai²,  Junjun Jiang³,  Jiayi Ma^1†

¹Wuhan University   ²Macau University of Science and Technology   ³Harbin Institute of Technology
^*Equal Contribution   ^†Corresponding Author

🔎 Method Overview

Motivation

Framework

Frequency Domain Comparison

🔧 Environment Setup

1. Clone this repository:

   git clone https://github.com/Linfeng-Tang/ControlFusion.git
   cd ControlFusion

2. Create a Conda environment (recommended):

   conda create -n controlfusion python=3.8 -y
   conda activate controlfusion

3. Install dependency packages:

   pip install -r requirements.txt

📂 Dataset Construction

please refer to genDateset,To simulate light degradation, use Lightroom Classic
Our dataset will be open sourced soon.

📥 Pre-trained Weights

Download the pretrained model Mask-DiFuser from Baidu Drive, and put the weight into pretrained_weights/.

🧪 Inference

You can use the test.py script we provide to fuse pairs of images. Please make sure you have downloaded the pre-trained weights. You can modify ControlFusion.py to select text/auto control by:

text_features = self.get_text_feature(text.expand(b, -1)).to(inp_img_A.dtype)
text_features = imgfeature

🚂 Train

You can use the train.py script we provide to train. Make sure you have organized your train dataset correctly.

📷 Results

Visualization of fusion results in different degraded scenarios

Generalization results in the real world

🕵️‍♂️ Detection

🎓 Citations

If our work is useful for your research, please consider citing and give us a star ⭐:

@inproceedings{Tang2024Mask-DiFuser,
  author={Linfeng Tang, Yeda Wang, Zhanchuan Cai, Junjun Jiang, and Jiayi Ma},
  title={ControlFusion: A Controllable Image Fusion Network with Language-Vision Degradation Prompts}, 
  booktitle={Advances in Neural Information Processing Systems},
  year={2025},
 }

🤝 Contact

Please feel free to contact: [email protected], [email protected]. We are very pleased to communicate with you and will maintain this repository during our free time.