How to calculate VMAF for mobile viewing scenarios?

[yhwu0423]

How to calculate VMAF by phone model?

[fifonik]

I'm not sure what did you mean.
Add reference & distorted files in the program, tick "VMAF" metric, untick other metrics, select "VMAF model", tick "Phone" checkbox and run calculation?

[yhwu0423]

Thank you for your reply! I'm actually using FFmpeg command line to calculate VMAF, not a GUI program.I want to know how to compute VMAF for mobile scenarios with FFmpeg. When I set phone_model=1 or 0, the results are the same, but setting enable_transform=true gives a different score. Could you explain the difference between these two options?

[yhwu0423]

When I use the GUI, I checked the "Phone" option, but the VMAF calculation failed/errored out.

[fifonik]

I tested it locally and the "Phone" checkbox is broken. Checking.

[fifonik]

As I see, ffmpeg's libvmaf does not have options phone_model and enable_transform any longer. So FFMetrics cannot do anything about this right now.

[yhwu0423]

Thank you for your reply!
When I run this command:

ffmpeg -i dist_720p_2M.mp4 -i ref_10s.mp4 -lavfi "[0:v]scale=1920:1080:flags=bicubic[dist];[1:v]scale=1920:1080:flags=bicubic[ref];[dist][ref]libvmaf=model=version=vmaf_v0.6.1\\:enable_transform=true:log_fmt=json:log_path=vmaf_720p_phone.json" -f null -

I get a higher VMAF score than the default model.
And according to the FFmpeg documentation, phone_model is deprecated and replaced by enable_transform.
FFmpeg doc

[fifonik]

Thanks. Looks like I will need to fix that...

[yhwu0423]

Thank you for looking into this!
If you need any more details or test cases to help fix it, feel free to let me know.
Looking forward to the update!

[fifonik]

I've just released a new beta version where this should be fixed.

[yhwu0423]

Thanks a lot for the quick fix!

[fifonik]

Did you check it? Does it work for you?

[yhwu0423]

Now I can calculate the VMAF for the phone model normally and get the results successfully. But I'd like to know: when calculating the phone model VMAF between a 720p video and the original 4K video, are both videos uniformly scaled to 1080p first?

[fifonik]

As per settings in ur screenshot ("Upscale to model's resolution" is ticked) both will be resized to 1080 (as per VMAF devs recommendations, see #190).
If the checkbox mentioned above is not ticked, then ref will not be touched, and distorted will be upscaled to ref's resolution (that is 4K in ur case).

P.S. You can turn on an option "Log ffmpeg commands to log file" and see what ffmpeg commands were used my FFMetrics