LightCtrl: Training-free Controllable Video Relighting

Yizuo Peng^1,2 · Xuelin Chen³ · Kai Zhang^1,* · Xiaodong Cun^{2, *}

¹ Tsinghua University     ²GVC Lab, Great Bay University     ³Adobe Research  

Openreview | PDF

Demo Video

Method Overview

We perform controllable video relighting with a user-provided light trajectory. Where we inject the light map on the noisy latent of VDM using the light map injection module. Then, in each denosing step, we design a geometry-aware relighting module to produce the relighted results frame-wise. Thus, the VDM can help to generate consistent video results with controllable lighting.

More Results

Input Video	Light Map	Relighted video

Get Started

1. Setup the repository and environment

git clone https://github.com/GVCLab/LightCtrl.git
cd LightCtrl
conda create -n LightCtrl python==3.10
conda activate LightCtrl
pip install -r requirements.txt

2. Download the pretrained model

IC-Light:🤗Huggingface ;

SD RealisticVision: 🤗Huggingface ;

Animatediff Motion-Adapter-V-1.5.3: 🤗Huggingface ;

CogVideoX-2b: 🤗Huggingface

3. 💡Relighting

Video relighting with user-defined light setting:

python relight.py --config "configs/animatediff_relight/music_girl.yaml"

LightCtrl also supports the CogVideoX backbone:

python cog_relight.py --config "configs/cog_relight/blackman.yaml"

Citation

If you find our work helpful, please leave us a star and cite our paper.

@inproceedings{ peng2026lightctrl,
title={LightCtrl: Training-free Controllable Video Relighting},
author={Yizuo Peng and Xuelin Chen and Kai Zhang and Xiaodong Cun},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=5ft8vd9rwc}
}

Acknowledgements

We are very grateful to the authors of Light-A-Video, AnimateDiff, CogVideoX, and IC-Light for the foundation of our code from their open-source code.