Shuai Liu, Quanmin Liang, Zefeng Li, Boyang Li, Kai Huang
School of Computer Science and Engineering, Sun Yat-sen University
Contact: liush376@mail2.sysu.edu.cn, liby83@mail.sysu.edu.cn, huangk36@mail.sysu.edu.cn
Sep 18, 2025: GaussianFusion has been accepted to NeurIPS 2025 (Spotlight).Oct 09, 2025: The source code of GaussianFusion has been publicly released.Oct 26, 2025: Update the implementation of V2-99 version.Oct 26, 2025: The checkpoint file of GaussianFusion has been uploaded.
End-to-end (E2E) autonomous driving aims to simplify the traditional modular pipeline by directly mapping multi-sensor inputs to driving actions through deep learning, enabling joint optimization and reducing system complexity. However, relying on a single sensor limits robustness in diverse scenarios, making multi-sensor fusion essential for reliable perception and robust driving policies.
Existing fusion strategies are mainly categorized as flatten fusion and bird’s eye view (BEV) fusion. Flatten fusion compresses sensor features into a shared latent space using attention mechanisms, offering flexibility but lacking explicit spatial grounding. BEV fusion projects multi-modal features into a common BEV coordinate, improving spatial understanding but incurring high computational and memory costs due to dense representations.
GaussianFusion introduces a novel Gaussian-based multi-sensor fusion framework for E2E autonomous driving. Our approach leverages sparse and interpretable 2D Gaussians to efficiently integrate multi-modal features from cameras and LiDARs. The dual-branch fusion pipeline is tailored for both scene reconstruction and motion planning, requiring only BEV semantic supervision. A cascade planning module further refines trajectory predictions through hierarchical Gaussian queries.
Extensive evaluations on both open-loop (NAVSIM) and closed-loop (Bench2Drive) benchmarks demonstrate the superior performance and robustness of GaussianFusion compared.
Visualization of predicted and ground-truth trajectories
Dataset cache for training.
bash ./scripts/training/run_train_metric_cache.sh
bash ./scripts/training/run_dataset_cache.shMetric cache for evaluation.
bash ./scripts/evaluation/run_metric_caching.shBefore starting training, ensure that the dataset cache is already prepared and the path settings are correct.
bash ./scripts/training/run_gaussianfusion.sh Before starting evaluation, ensure that the metric cache is already prepared and the path of pt file is correct.
bash ./scripts/evaluation/run_gaussianfusion.sh| Model | NC | DAC | TTC | EP | PDMS | ckpt |
|---|---|---|---|---|---|---|
| GaussianFusion (V2-99) | 98.6 | 98.6 | 95.2 | 89.7 | 92.6 | ckpt |
If any part of our paper or code is helpful to your research, please consider citing our work 📝 and give us a star ⭐. Thanks for your support!
@article{liu2025gaussianfusion,
title={GaussianFusion: Gaussian-Based Multi-Sensor Fusion for End-to-End Autonomous Driving},
author={Liu, Shuai and Liang, Quanmin and Li, Zefeng and Li, Boyang and Huang, Kai},
journal={arXiv preprint arXiv:2506.00034},
year={2025}
}This is an enhanced implementation of GaussianFusion which is slightly different from the paper version. The code is built upon the great open-source efforts of:
For questions, feel free to reach out: 📧 liush376@mail2.sysu.edu.cn / liby83@mail.sysu.edu.cn /