feat(amf): add max_au_size option to prevent FEC bypass on Wi-Fi

[pollopopo]

Description

Adds a new optional AMF encoder parameter amd_max_au_size that exposes the existing FFmpeg/AMF maximum Access Unit size option to Sunshine users. This is not a silver bullet — it is an additional tool in the toolbox for users who have exhausted other options and are still experiencing stream instability, particularly on problematic Wi-Fi connections.

Most users will never need this setting. The existing rate control modes (VBR_LATENCY, CBR) and other options (enforce_hrd, vbaq, preanalysis) should be the first line of troubleshooting. This option is intended as a last resort for difficult network conditions where standard rate control cannot prevent occasional oversized frames from disrupting the stream.

Background

Sunshine uses FEC (Forward Error Correction) to protect video frames against packet loss. FEC has a hard limit: each block supports max 255 data shards, with max 4 blocks per frame. When an encoded frame exceeds this limit (~889 KB at default settings), FEC is disabled for that frame:

// stream.cpp line 1373
if (fec_blocks_needed > MAX_FEC_BLOCKS) {
    BOOST_LOG(warning) << "Skipping FEC for abnormally large encoded frame...";
    fecPercentage = 0;
}

Additionally, large encoded frames produce large packet bursts that can overwhelm Wi-Fi TX queues, causing packet loss even when the link has sufficient average bandwidth.

Standard rate control modes (VBR, CBR) treat bitrate targets as soft constraints and can still produce occasional frame size spikes during high-complexity scenes. For most users on decent networks, these spikes are harmless. But on congested or unreliable Wi-Fi, they can be the difference between a stable and unstable stream.

What this PR does

Exposes the AMF max_au_size encoder option, which sets a hard, hardware-enforced cap on the maximum encoded frame size in bits. This maps to AMF_VIDEO_ENCODER_MAX_AU_SIZE (H.264) and AMF_VIDEO_ENCODER_HEVC_MAX_AU_SIZE (HEVC). AV1 does not support this option and has been excluded.

• Disabled by default — no change to existing behavior for any user
• Optional — uses std::optional<int>, completely ignored when not set (FFmpeg default: -1)
• Zero overhead — the cap is enforced by the AMF hardware encoder, not in software
• Follows existing patterns — same implementation approach as amd_enforce_hrd, amd_vbaq, etc.

When to use it

This setting is for users who:

• Are streaming over Wi-Fi and experiencing dropped frames or stuttering
• Have already tried adjusting bitrate, rate control mode, FEC percentage, and other standard settings
• Are still seeing instability due to occasional frame size spikes

It is not a replacement for proper rate control configuration and network optimization.

Trade-offs

When the cap is hit, the encoder is forced to reduce quality for that frame, which may cause brief visual quality drops. The value also effectively limits the maximum bitrate regardless of what is configured in the client. Users should set it as high as their network reliably handles — not lower than necessary.

Choosing a value

The value is in bits. It controls the maximum size of any single encoded frame. Start with 800000 (~97 KB per frame, ~46 Mbps at 60fps) and increase until you find the highest value that remains stable on your network.

Value	Max frame size	Max bitrate at 60fps
800000	~97 KB	~46 Mbps
1000000	~122 KB	~58 Mbps
1600000	~195 KB	~93 Mbps

Screenshot

New "AMF Max AU Size (bits)" field appears in the AMD AMF Encoder > Rate Control Settings section of the Web UI.

Issues Fixed or Closed

Roadmap Issues

Type of Change

• feat: New feature (non-breaking change which adds functionality)
• fix: Bug fix (non-breaking change which fixes an issue)
• docs: Documentation only changes
• style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
• refactor: Code change that neither fixes a bug nor adds a feature
• perf: Code change that improves performance
• test: Adding missing tests or correcting existing tests
• build: Changes that affect the build system or external dependencies
• ci: Changes to CI configuration files and scripts
• chore: Other changes that don't modify src or test files
• revert: Reverts a previous commit
• BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

• Code follows the style guidelines of this project
• Code has been self-reviewed
• Code has been commented, particularly in hard-to-understand areas
• Code docstring/documentation-blocks for new or existing methods/components have been added or updated
• Unit tests have been added or updated for any new or modified functionality

AI Usage

• None: No AI tools were used in creating this PR
• Light: AI provided minor assistance (formatting, simple suggestions)
• Moderate: AI helped with code generation or debugging specific parts
• Heavy: AI generated most or all of the code changes

[Copilot]

Pull request overview

Adds a new AMD AMF encoder configuration option intended to cap maximum encoded frame size to avoid oversized frames bypassing FEC on lossy links (e.g., Wi‑Fi).

Changes:

• Introduces amd_max_au_size as a new optional AMD video encoder setting in config parsing/defaults.
• Wires the setting into AMF encoder option dictionaries (H.264/HEVC/AV1) and exposes it in the Web UI.
• Documents and localizes the new setting.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/config.h	Adds std::optional<int> amd_max_au_size to AMD video config struct.
src/config.cpp	Initializes default and parses amd_max_au_size from config vars.
src/video.cpp	Adds max_au_size option mapping for AMF codec configurations.
src_assets/common/assets/web/config.html	Adds default amd_max_au_size field for the UI’s default config.
src_assets/common/assets/web/configs/tabs/encoders/AmdAmfEncoder.vue	Adds a numeric input for amd_max_au_size in the AMD AMF tab.
src_assets/common/assets/web/public/assets/locale/en.json	Adds label/description strings for the new option.
docs/configuration.md	Documents amd_max_au_size and provides an example.

[Copilot]

This parses amd_max_au_size as an optional int but does not enforce non-negative values, and it will treat an explicit "0" as a configured value (so it will still be passed through to FFmpeg). If the intent is “0/empty disables”, consider normalizing values <= 0 to nullopt here (or validating a non-negative range) so downstream option application can skip setting the codec option entirely.

Suggested change

	int_f(vars, "amd_max_au_size", video.amd.amd_max_au_size);
	int_f(vars, "amd_max_au_size", video.amd.amd_max_au_size);
	if (video.amd.amd_max_au_size && *video.amd.amd_max_au_size <= 0) {
	video.amd.amd_max_au_size.reset();
	}

[Copilot]

The description says the value is in bytes and recommends 800000, but FFmpeg’s AMF "max_au_size" option for h264_amf/hevc_amf is in bits (and av1_amf doesn’t support this option at all). Please align the UI text with the actual behavior (units, recommended value, and which codecs it applies to) to avoid users configuring an ineffective or failing setting.

Suggested change

	"amd_max_au_size_desc": "Maximum encoded frame size in bytes (0 or empty = disabled). Caps the largest frame the encoder can produce, preventing oversized frames from exceeding FEC limits. Recommended value: 800000 for Wi-Fi streaming with CQP mode.",
	"amd_max_au_size_desc": "Maximum encoded frame size in bits for H.264/HEVC AMF encoders (0 or empty = disabled). Caps the largest frame the encoder can produce, preventing oversized frames from exceeding FEC limits. Recommended value: 6400000 bits (equivalent to 800000 bytes) for Wi-Fi streaming with CQP mode.",

[Copilot]

This section documents the cap in bytes and implies it applies to all AMF codecs, but in FFmpeg the exposed "max_au_size" AVOption is for h264_amf/hevc_amf (in bits) and av1_amf doesn’t expose an equivalent option. Please update the docs to match the actual supported codecs and units, and clarify the default/disable behavior (e.g., unset vs 0 vs -1).

Suggested change

	Maximum encoded frame size (Access Unit) in bytes. When set to a non-zero value, the
	encoder will cap the maximum frame size. This prevents oversized frames from exceeding
	FEC shard limits (DATA_SHARDS_MAX=255), which otherwise causes FEC to be skipped for
	those frames. This is especially important for Wi-Fi streaming where packet loss on
	large unprotected frames leads to dropped frames and stuttering.
	@note{This option only applies when using amdvce [encoder](#encoder).}
	@note{Works with all rate control modes (CQP, CBR, VBR). Most beneficial with CQP
	where frame sizes are otherwise unbounded.}
	</td>
	</tr>
	<tr>
	<td>Default</td>
	<td colspan="2">@code{}
	@endcode</td>
	</tr>
	<tr>
	<td>Example</td>
	<td colspan="2">@code{}
	Maximum encoded frame size (Access Unit) in <strong>bits</strong>. When set to a
	positive, non-zero value, Sunshine configures the AMF encoder to cap the maximum frame
	size. This prevents oversized frames from exceeding FEC shard limits
	(DATA_SHARDS_MAX=255), which otherwise causes FEC to be skipped for those frames. This
	is especially important for Wi-Fi streaming where packet loss on large unprotected
	frames leads to dropped frames and stuttering.
	@note{This option is forwarded to FFmpeg's @code{max_au_size} AVOption and only
	applies when using the amdvce [encoder](#encoder) with H.264 or HEVC
	(@code{h264_amf}/@code{hevc_amf}). It is not available for AV1 (@code{av1_amf}).}
	@note{Leave this option unset to use the encoder default (no explicit cap). Non-positive
	values are treated as "no limit" and should normally be avoided in favor of leaving the
	option unset.}
	@note{Works with all rate control modes (CQP, CBR, VBR). Most beneficial with CQP where
	frame sizes are otherwise unbounded.}
	</td>
	</tr>
	<tr>
	<td>Default</td>
	<td colspan="2">@code{}
	# unset: do not configure max_au_size, use encoder default (no explicit cap)
	# amd_max_au_size is disabled unless you set a positive value
	@endcode</td>
	</tr>
	<tr>
	<td>Example</td>
	<td colspan="2">@code{}
	# limit access unit size to 800000 bits

[Copilot]

FFmpeg’s av1_amf encoder does not expose an AVOption named "max_au_size" (unlike h264_amf/hevc_amf). Adding this option here means the cap won’t work for AV1 and may cause avcodec_open2 to fail with an unknown option when the setting is configured. Consider removing this option from the av1_amf option set (or gating it so it’s only applied for codecs that support it) and updating the UI/docs accordingly.

Suggested change

{"max_au_size"s, &config::video.amd.amd_max_au_size},

[Copilot]

The AMF "max_au_size" AVOption in FFmpeg for h264_amf/hevc_amf is specified in bits (and FFmpeg commonly uses -1 as the disable sentinel). The UI/docs describe this value as bytes and recommend 800000, which will be interpreted as ~100KB if passed through unchanged. Either convert bytes->bits before setting the codec option, or update the config/UI/docs to clearly use bits (and document the disable behavior).

Suggested change

	{"max_au_size"s, &config::video.amd.amd_max_au_size},
	{"max_au_size"s, []() {
	// Config/UI use bytes; FFmpeg AMF option expects bits. Negative disables (maps to -1).
	const int value_bytes = config::video.amd.amd_max_au_size;
	if (value_bytes < 0) {
	return -1;
	}
	return value_bytes * 8;
	}},

[Copilot]

Same as above: the FFmpeg "max_au_size" option is in bits. Passing the config value through as-is while documenting bytes will lead to an incorrect cap. Consider converting bytes->bits (and treating 0/empty as unset) before setting this option.

Suggested change

	{"max_au_size"s, &config::video.amd.amd_max_au_size},
	{"max_au_size"s, []() {
	const auto bytes = config::video.amd.amd_max_au_size;
	if (bytes <= 0) {
	return 0;
	}
	return bytes * 8;
	}},

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud