Facial Spatiotemporal Graphs for rPPG

A self-contained Python package containing an packaged facial spatiotemporal graph processing module similar to that used in the paper Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement. For the full experimental code-base including more comprehensive pre- and post-processing used to obtain the results in the paper please check out the experiments branch.

🔧 Installation

Prerequisites

• Python 3.10 or higher.
• CUDA-compatible GPU.

Install from GitHub

You can easily install this for use in existing projects:

pip install git+https://github.com/csiro/facial-stgraph-rppg.git@package

💻 Quick Start

If you want to explore the examples shown this repository in a separate project we recommend using uv to create a virtual environment and install dependencies:

git clone https://github.com/csiro/facial-stgraph-rppg.git@package
cd facial-stgraph-rppg
uv venv .venv --python 3.10
source .venv/bin/activate
uv pip install -e ".[examples]"

Basic Usage

The simplest way to use the package:

import torch
import facial_stgraph_rppg as stgraph

# Create video processor with custom parameters
processor = stgraph.VideoProcessor(
    min_detection_confidence=0.7,    # Higher confidence threshold
    min_tracking_confidence=0.8,     # More stable tracking
    device="cuda",                   # Use GPU acceleration
    degree_masking_threshold=90.0    # Stricter orientation filtering
)

# Load your video frames
frames = torch.randn(200, 720, 1280, 3)  # HD video frames

# Process the video
Xp, Xn, _, Xc, Aw = processor(frames) # STgraph = (Xc, Aw)
print(f"Input: {frames.shape}")
print(f"Output: {Xc.shape}, {Aw.shape}")  # Should be [200, V, 3], [1, V, V]

You may also use pre-computed 3D landmarks from the MediaPipe FaceMesh module.

import torch
import facial_stgraph_rppg as stgraph

# Create video processor with custom parameters
processor = stgraph.VideoProcessor(
    min_detection_confidence=0.7,    # Higher confidence threshold
    min_tracking_confidence=0.8,     # More stable tracking
    device="cuda",                   # Use GPU acceleration
    degree_masking_threshold=90.0    # Stricter orientation filtering
)

# Load your video frames
frames = torch.randn(200, 720, 1280, 3)  # HD video frames

# 3D landmark detection using the MediaPipe FaceMesh pipeline
detector = stgraph.MediaPipeLandmarkDetector()
landmarks_txyz = detector(frames)
landmarks_txyz.shape # [T,V,3]

# Process the video
Xp, Xn, _, Xc, Aw = processor(
    frames=frames, 
    landmarks_txyz=landmarks_txyz
) # STgraph = (Xc, Aw)
print(f"Input: {frames.shape}")
print(f"Output: {Xc.shape}, {Aw.shape}")  # Should be [200, V, 3], [1, V, V]

Please see the examples/ directory for basic and advanced usage examples and visualizations.

Note: The repository’s code license does not apply to the example video examples/34d3d5ef-b98f-46e1-b481-2249752423c4.mp4. The example video is licensed for non-commercial research and testing with this repository only. Redistribution, publication, or inclusion in datasets/models is not permitted without explicit written permission from the owner Sam Cantrill (sam.cantrill@gmail.com).

📜 Citation

If you find this useful please cite our work.

@article{cantrill2025facialstgraphrppg,
      title={Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement}, 
      author={Sam Cantrill and David Ahmedt-Aristizabal and Lars Petersson and Hanna Suominen and Mohammad Ali Armin},
      year={2025},
}