regression: bad video quality now with -p vaapi

[rfht]

Video quality when using vaapi seems to have deteriorated with commit ace0882

[morgant]

"Thinking out loud" here to organize my thoughts:

• Related to Issue Gradually increasing video & audio desync over period of a stream #5, since we're explicitly setting the -thread_queue_size ffmpeg advanced option for each input & output, could that be introducing desync? It's described thusly (copypasta here for easier reference since there are no direct permalink anchors):

  -thread_queue_size size (input/output)

  For input, this option sets the maximum number of queued packets when reading from the file or device. With low latency / high rate live streams, packets may be discarded if they are not read in a timely manner; setting this value can force ffmpeg to use a separate input thread and read packets as soon as they arrive. By default ffmpeg only does this if multiple inputs are specified.

  For output, this option specified the maximum number of packets that may be queued to each muxing thread.

  I'm not sure whether I'm interpreting it correctly, but it may be that our "low latency / high rate live streams" use-case should be discarding packets that can't be read quickly enough instead of ensuring they're queued.

• I see that we lost setting $video_output_fps in the implementation of missing VAAPI support #8, at least when using VA-API. Could that be related?

• I know we set -threads, would it also be worth overriding -filter_threads and/or -complex_filter_threads? They both default to the number of CPUs, but we know we're already running at least one X11 display video input capture, up to two (none, mic, and/or mix) sndio audio input captures, and potentially an audio mix/filter. Would it be useful to calculate something like (pseudocode) CPU_COUNT - AUDIO_INPUTS_COUNT -1 || 1 to limit too many threads being used?

[morgant]

I tested this and found the degradation of video quality exists even when not using VA-API (see #8). I was only getting ~6-7fps with x264 encoding, though I can usually maintain 25fps.

I have a WIP branch I'm testing with and found that reverting the move of video capture to the left side of the pipe resolves the performance issue. My hypothesis is that moving the x11grab video input (the new $VIDEO_IN) to the first process and leaving the encoding (the new $VIDEO_CONV) on the second process means that the raw video frames from x11grab are being passed in nut format through the pipe, to then be encoded. That must remove any optimizations which ffmpeg has to fetch & encode the video frames directly (or utilize VA-API, if enabled).

I'm open to other other thoughts & concerns. I haven't created a PR yet, but I'm happy to do so if anyone wants to test. (I reverted the functionality, but retained the nice new variables.)

[morgant]

I've now prepared PR #12 to address this. Feel free to test.

[rfht]

Thanks, solved with #12