Controllable Human-centric Keyframe Interpolation with Generative Prior

Zujin Guo¹, Size Wu¹, Zhongang Cai², Wei Li¹, Chen Change Loy¹

¹S-Lab, Nanyang Technological University, ²SenseTime Research

NeurIPS 2025

We introduce PoseFuse3D Keyframe Interpolator (PoseFuse3D-KI), a novel framework that integrates 3D human guidance signals into the diffusion process for Controllable Human-centric Keyframe Interpolation (CHKI).

📖 For more visual results of PoseFuse3D-KI, go checkout our project page.

---

🚩 News

• 2025.12.31: 🚀 Inference code and model checkpoints are released!
• 2025.09.17: 🎉 PoseFuse3D-KI has been accepted by NeurIPS 2025!

⚙️ Installation

# Create a conda environment
conda create -n pf3d python=3.10 -y
conda activate pf3d

# Install PyTorch
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1

# Install dependencies
pip install -r requirements.txt

📂 Preparation

1. SMPL-X Models: Download the SMPL-X body models and place them in ./data/smplx_files/smplx.
2. Checkpoints: Download our model checkpoints and place them in the ./ckpt directory.

The expected file structure should be as follows:

PoseFuse3D-KI/
├── ckpt/ 
│   ├── pf3d/
│   │   ├── control_model.safetensors
│   │   └── pytorch_lora_weights.safetensors
│   └── pf3d_flf2v/
│       ├── control_model.safetensors
│       └── pytorch_lora_weights.safetensors
├── data/
│   └── smplx_files/
│       └── smplx/ 
│           ├── MANO_SMPLX_vertex_ids.pkl
│           ├── SMPL-X__FLAME_vertex_ids.npy
│           ├── SMPLX_NEUTRAL.pkl
│           ├── SMPLX_to_J14.pkl
│           ├── SMPLX_NEUTRAL.npz
│           ├── SMPLX_MALE.npz
│           └── SMPLX_FEMALE.npz

🚀 Demo

We provide two interpolation examples in the ./examples directory. The structure of an example is as follows:

PoseFuse3D-KI/
└── examples/
    └── example1/
        ├── frames/
        │   ├── frame_0000.png      # First keyframe
        │   └── frame_0024.png      # Last keyframe
        ├── pose/
        │   ├── frame_0000.png      # 2D pose guidance
        │   └── ...
        └── smplx_ann/              # 3D SMPL-X annotations
            ├── img_cs/
            ├── meta/
            └── smplx/

To run the interpolation demo, use the following command:

# Standard interpolation
sh demo.sh

# Interpolation with FLF2V 
sh demo_flf2v.sh

The results will be saved in ./examples/example1/output.

📝 Citation

If you find our work useful for your research, please consider citing:

@inproceedings{guo2025controllable,
    title={Controllable Human-centric Keyframe Interpolation with Generative Prior},
    author={Guo, Zujin and Wu, Size and Cai, Zhongang and Li, Wei and Loy, Chen Change},
    booktitle={Advances in Neural Information Processing Systems},
    year={2025}
}