Skip to content

Feature Request: Compact L1 Export (CSV/Raw) to reduce IO/Storage overhead #374

@cryptochrome

Description

@cryptochrome

Context

I am the maintainer of dovi_convert, a tool that automates Profile 7 to 8.1 conversion. I use dovi_tool to analyze the RPU of full-length MKV files (typically 170k+ frames) to detect luminance expansion in the FEL.

The Problem

Currently, to analyze RPU metadata, we must use dovi_tool export ... -d all=dump.json.
While comprehensive, the JSON output is extremely verbose for L1 analysis.

  • Expansion Factor: ~16x (Binary RPU -> JSON)
  • Real World Impact: A 35MB RPU binary for a standard movie results in a ~560MB - 1GB JSON file.
  • Performance Cost:
    1. Disk I/O: Writing 1GB of text to /tmp is a significant bottleneck on slower storage/networks.
    2. Memory: Downstream tools (like Python) require massive RAM (3GB+) to parse such large JSON files using standard libraries.

Feature Request
Could you please consider adding a lightweight export mode that outputs only the frame-level L1 metadata in a compact format (CSV or similar)?

Proposed Interface:
dovi_tool export -i input.rpu --mode l1-csv > output.csv

Desired Output (Example):
A headerless or simple CSV structure containing just the essential L1 metrics per frame:

frame_index,max_pq,avg_pq,min_pq
0,3050,400,0
1,3055,402,0
...

Impact

  • Size Reduction: 560MB -> ~5MB (99% reduction).
  • Speed: Removes the disk I/O bottleneck and memory usage entirely.
  • Integration: Trivial to parse line-by-line in scripts without memory overhead.

Thank you for the amazing tool!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions