-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Problem Summary
While processing the FaceOLAT dataset for foreground segmentation/matting tasks, I discovered that the fullbright frame index is inconsistent across different subjects and takes. Specifically, frame 188 (e.g., {expressionID}.000188.exr) - which I assumed to be the fullbright reference frame - does not contain full illumination for many subjects.
Expected Behavior
According to the dataset documentation, each take contains 350 lighting conditions. I expected a consistent frame index (e.g., frame 188) to represent the fullbright condition (all lights on) across all subjects, cameras, and takes.
Actual Behavior
The fullbright frame appears at different indices for different subjects. In some cases, the first 16 frames appear to be fullbright instead of frame 188.
Specific Example
| Path | Expected | Actual |
|---|---|---|
007/007/ID40007/Cam01/C006/ID40007.000188.exr |
Fullbright frame | NOT fullbright (single light or partial illumination) |
007/007/ID40007/Cam01/C006/ID40007.000001.exr to ID40007.000015.exr |
Single light frames | Appear to be fullbright |
Impact
This inconsistency affects downstream processing tasks such as:
- Background matting/segmentation - Using incorrect frames produces wrong masks
- Relighting pipelines - Incorrect fullbright reference affects light transport estimation
- Any processing assuming consistent frame ordering
Questions
- Is there a documented specification for which frame index corresponds to the fullbright condition?
- Does the fullbright frame index vary by subject, expression, or capture session?
- Is there metadata available that maps each frame to its lighting condition?
- Could you provide guidance on how to reliably identify the fullbright frame for each take?