Find muxer options in VideoEncoder to test fragmented mp4

[Dan-Flores]

This PR adds logic to sortCodecOptions to find and set muxer options (check available options with ffmpeg -h muxer=mp4)

With these changes, we can now test that fragment related movflag options are utilized correctly.
But, without a 'streaming' video encoder, the benefits of fragmented mp4's are still not unlocked:

• Fragmented MP4s are useful because they allow the video file to be consumed before it is closed.
• However, since our encoder expects all frames to be encoded at once, the fragments are not flushed to become available to consume until the end of the encoding process, so we cannot access the video before encoding is completed.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1216

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 50aae18 with merge base 6f10da0 ():

NEW FAILURE - The following job has failed:

• Lint / mypy (3.12) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Dan-Flores]

Without this conversion, we interpret the positive returned integer as a random low level error unrelated to the encoding process (ex. 24 raises EMFILE: too many open files). It might just be a bug in the FFmpeg implementation that we can work around.

Since FFmpeg errors are negative, I think its fine to set the status to successful here.

[mollyxu]

Thanks for providing context! Would we want to also add a comment on how ffmpeg errors are negative and link to source code?

[Dan-Flores]

Updated!

[NicolasHug]

A few Qs and nits but LGTM, thanks @Dan-Flores

[NicolasHug]

Any reason not to use AV_OPT_SEARCH_CHILDREN here? It's used above for fmtOpt

[Dan-Flores]

We only use AV_OPT_SEARCH_CHILDREN when an AVClass has children to iterate over.
I checked that the AVFormatContext class defines child_next and child_class_iterate to iterate through child fields:
https://github.com/FFmpeg/FFmpeg/blob/release/8.0/libavformat/options.c#L128-L137

While the muxer class for MOV does not define these functions:
https://github.com/FFmpeg/FFmpeg/blob/release/8.0/libavformat/movenc.c#L132-L137

[NicolasHug]

Where do we error? By "we" you mean TC, or FFmpeg (i.e. is this limitation enforced by us, or by FFmpeg)?

[Dan-Flores]

This is a torchcodec limitation, I was able to decode the truncated video with FFmpeg 4. I'll update the reason message.

The error we get is Invalid data found when processing input, which means we are hitting AVERROR_INVALIDDATA (see errors), which we do not handle, we only check for AVERROR_EOF:

torchcodec/src/torchcodec/_core/SingleStreamDecoder.cpp

Lines 265 to 272 in 6f10da0

	if (status == AVERROR_EOF) {
	break;
	}
	STD_TORCH_CHECK(
	status == AVSUCCESS,
	"Failed to read frame from input file: ",
	getFFMPEGErrorStringFromErrorCode(status));

Since this error only occurs on FFmpeg 4, I think it should be ok to leave as is.

[NicolasHug]

Use torch.testing.assert_equal(..., rtol=0, atol=0), it provides better error messages than torch,equal.

[NicolasHug]

This is a fun test!
Just to make sure, did you verify that it breaks if we don't pass the extra_option flags?

[Dan-Flores]

Yes, it fails to open the file when using extra_options={}:
RuntimeError: Could not open input file: .../fragmented_output.mp4 Invalid data found when processing input

Hopefully this fun test does not introduce flakiness!