Skip to content

Commit f77825a

Browse files
staging-devin-ai-integration[bot]streamkit-devinstreamer45
authored
feat: add AAC encoder native plugin with MP4/MoQ support (#261)
* feat: add AAC encoder native plugin with MP4/MoQ support Implement an AAC-LC encoder as a native plugin using shiguredo_fdk_aac 2025.1.1, keeping non-royalty-free codec dependencies out of the core. Plugin (plugins/native/aac-encoder/): - NativeProcessorNode impl: f32→i16 PCM conversion, 1024-sample framing, configurable bitrate (default 128 kbps), content_type and metadata preservation via BinaryWithMeta packets. Plugin SDK C ABI (v7, backward-compatible with v6): - New CPacketType::BinaryWithMeta variant and CBinaryPacket struct to preserve content_type and metadata across the native plugin boundary. - Plugin host accepts both v6 and v7 plugins. Core types: - Add AudioCodec::Aac variant. MP4 muxer: - Explicit Aac match arms in content type and sample entry builders. - New audio_codec config field for codec override. MoQ transport (push + peer): - AAC in moq_accepted_media_types(). - catalog_audio_codec() / resolve_audio_codec() / parse_audio_codec_config() helpers mirroring the video codec pattern. - audio_codec config field on MoqPushConfig and MoqPeerConfig. Build system: - just build-plugin-native-aac-encoder target. - lint-plugins / fix-plugins / build-plugins-native entries. Sample pipelines: - oneshot/aac_encode.yml (audio-only AAC in MP4) - oneshot/mp4_mux_aac_h264.yml (AAC + H264 in MP4) - dynamic/moq_aac_mixing.yml (MoQ broadcasting with mixing + gain) Signed-off-by: Devin AI <devin@cognition.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: downgrade BinaryWithMeta for v6 plugins, fix audio_codec deserialization Address two bugs found by Devin Review: 1. BinaryWithMeta (discriminant 10) was sent to v6 plugins that only understand discriminants 0-9, causing packet drops. Fix: store the plugin API version in InstanceState and call downgrade_binary_with_meta() before forwarding to v6 plugins. This converts to plain Binary, preserving the raw bytes while dropping the content_type/metadata that v6 cannot interpret. 2. Mp4MuxerConfig.audio_codec was typed as Option<AudioCodec>, but the AudioCodec enum has no serde rename_all attribute, so YAML values like 'aac' (lowercase) failed deserialization. Fix: change the field to Option<String> with a case-insensitive parse helper, consistent with MoqPeerConfig/MoqPushConfig. Includes regression tests for the downgrade logic. Signed-off-by: Devin AI <devin@cognition.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: accept mono input in AAC encoder, upmix to stereo The Opus decoder outputs mono (1 channel) but the AAC encoder previously only accepted stereo (2 channels), causing an incompatible connection error in the graph builder. Fix: accept both mono and stereo input on the 'in' pin. Mono samples are duplicated to both L/R channels before encoding, since the FDK AAC library (shiguredo_fdk_aac) hardcodes stereo output. Signed-off-by: Devin AI <devin@cognition.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: address code review feedback for AAC encoder PR - Split wildcard Aac | _ pattern into explicit arms with tracing::warn for unrecognised future audio codecs (Critical #1) - Parameterize DEFAULT_AUDIO_FRAME_DURATION_US by codec: Opus 20ms, AAC ~21.333ms via const fn helpers (Suggestion #2) - Compute AAC timestamps from frame count to avoid truncation drift: sequence * 1024 * 1_000_000 / 48_000 (Suggestion #3) - Document Binary vs EncodedAudio semantic mismatch in AAC encoder output pin (Suggestion #4) - Bundle video/audio codec into MediaCodecConfig struct for handle_pin_management (Suggestion #5) - Deduplicate parse_audio_codec_config: mp4.rs delegates to shared implementation in moq/constants.rs (Nit #1) - Document 960→1024 mixer/encoder frame size interaction and rewrite moq_aac_mixing.yml as documented placeholder (Nit #2) Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: inline parse_audio_codec_config in mp4.rs to avoid moq feature dependency The mp4 feature does not depend on moq, so importing from transport::moq::constants would break --features mp4 builds. Inline the trivial parsing logic directly in mp4.rs instead. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: use map_or for parse_mp4_audio_codec_config (clippy) Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: address second round of code review feedback - Add TODO comments for MoqPullNode Opus hardcoding (blocked by Binary→EncodedAudio C ABI gap) - Add video_codec config field to MP4 muxer for accurate pre-connection MIME hint (mirrors existing audio_codec field) - Add make_dynamic_output_pin AAC test verifying AudioCodec::Aac is threaded through to audio output pins - Rewrite moq_aac_mixing.yml as runnable pipeline with MP4 muxer sink instead of broken placeholder - Set video_codec: h264 in mp4_mux_aac_h264.yml for correct hint Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: accept Binary in MP4 muxer input pins + fix mixed-source oneshot pipelines - Add PacketType::Binary to MP4 muxer accepted input types so native plugins (which output Binary via C ABI) can connect directly. - Fix oneshot engine to detect generator root nodes (e.g. colorbars) even when http_input nodes are present, enabling mixed-source pipelines like AAC+H264 MP4 mux. - Fix mp4_mux_aac_h264.yml: use explicit pin mapping (in/in_1) and add num_inputs: 2 for dual-stream muxing. - Fix clippy single_option_map lint on parse_mp4_video_codec_config. Validated end-to-end: - aac_encode.yml: AAC-LC 48kHz stereo 128kbps in MP4 container - mp4_mux_aac_h264.yml: H.264 640x480 + AAC-LC stereo in MP4 Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: use fragmented MP4 for browser MSE playback + drift-free duration_us - Change aac_encode.yml from mode: file to mode: stream so the MP4 output contains mvex/moof atoms required by Media Source Extensions. - Compute duration_us from frame count (next_timestamp - this_timestamp) instead of using the truncated AAC_FRAME_DURATION_US constant, making duration consistent with the drift-free timestamp computation. - Remove unused AAC_FRAME_DURATION_US constant. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: MSE playback issues for AAC+H264 pipeline + MoQ YAML syntax - Fix MSE codec string mismatch: OpenH264 at 640x480 outputs Level 3.0 (avc1.42c01e), not Level 3.1 (avc1.42c01f). MSE is strict about this match and rejects the init segment when codecs don't match. - Fix moq_aac_mixing.yml: use dot syntax (moq_peer.audio/data) instead of bracket syntax (moq_peer[audio/data]) for dynamic pin references. - Improve classify_packet docstring to document all handled packet types. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: MSE playback + MoQ AAC pipeline issues - Initialise video_codec from config (matching audio_codec pattern) so the muxer uses H264 even when type resolution is unavailable. Previously video_codec was hardcoded to Av1, causing the init segment to contain an av01 track instead of avc1. - Fix placeholder AVC1 sample entry profile_compatibility (0 → 0xC0) to match the SPS constraint flags in the placeholder NAL unit. - Fix moq_aac_mixing.yml: replace unsupported bracket syntax (mic_gain[in_0]) with simple array syntax so Needs::Multiple auto-generates in_0/in_1 by index. - Add codec detection tracing for easier debugging. - Add regression tests for all three fixes. Signed-off-by: Devin AI <devin@streamkit.dev> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix(mp4): defer first fMP4 flush when inputs still open in skip-classification mode In skip-classification mode (dual-input with explicit dimensions), the safety cap on FMP4_FIRST_FLUSH_DEFER_CAP could force-flush an init segment before all expected tracks had produced data. When the audio path processes data much faster than the video path (e.g. file-based audio vs. a video generator that needs font initialization), the cap would trigger an audio-only init segment missing the expected h264 track, causing Chrome MSE to reject it with: 'Initialization segment misses expected h264 track' The fix checks whether input channels are still open before applying the safety cap. As long as channels remain open, a slow-starting track may still produce data, so the flush is deferred. Once all channels close, the cap fires normally to handle genuinely misconfigured pipelines. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * feat(plugin-sdk): add EncodedAudio discriminant to native plugin C ABI Add CPacketType::EncodedAudio (= 11) to the native plugin C ABI, allowing plugins to declare EncodedAudio output types (e.g. AAC) that are compatible with MoQ transport nodes. The codec name is carried in the existing custom_type_id pointer field (e.g. "aac", "opus") to preserve CPacketTypeInfo struct layout and maintain ABI compatibility with v6/v7 plugins. Also: - Bump NATIVE_PLUGIN_API_VERSION to 8 - Update AAC encoder plugin to declare EncodedAudio(Aac) output - Add CAudioCodec enum for documentation/future use - Add secondary hard cap (FMP4_SKIP_CLASS_HARD_CAP = 30000) for skip-classification fMP4 flush deferral to prevent unbounded memory growth from pathological misconfiguration - Create moq_aac_echo.yml sample pipeline for AAC echo over MoQ - Remove outdated MoQ AAC limitation comment from moq_aac_mixing.yml Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * style: rustfmt formatting for EncodedAudio conversions Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * feat(moq-peer): add subscriber_audio_codec config for transcoding pipelines Add a new subscriber_audio_codec parameter to MoqPeerConfig that controls the subscriber-side MoQ catalog codec independently from the publisher output pin type (audio_codec). This enables transcoding pipelines where the publisher sends one codec (e.g. Opus) but the pipeline re-encodes to another (e.g. AAC) before feeding it back to subscribers. Without this separation, audio_codec controlled both the output pin type AND the catalog codec, causing type mismatches in the graph builder. Also fixes moq_aac_mixing.yml: - Replace non-existent path/audio_only fields with correct gateway_path/input_broadcasts/output_broadcast/allow_reconnect - Remove dead-end mp4_muxer node; feed AAC directly back to moq_peer for MoQ streaming - Add client section for browser WebTransport connection Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * style: rustfmt formatting for subscriber_audio_codec Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix(moq-peer): use publisher codec for dynamic output pins Dynamic output pins carry data FROM the publisher, so they must use the publisher's audio_codec — not subscriber_audio_codec. Without this fix, non-primary broadcast output pins (created at runtime via handle_pin_management) would be incorrectly typed with the subscriber codec in transcoding pipelines. Also fixes misleading FMP4_SKIP_CLASS_HARD_CAP comment: 30,000 samples ≈ 10 minutes of audio at typical AAC rates, not seconds. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix(plugin-sdk): use EncodedAudio discriminant in macro metadata generation The native_plugin_entry! and native_source_plugin_entry! macros were using CPacketType::Binary as the fallback for non-Opus EncodedAudio variants (e.g. AAC). This caused the host to read the plugin's output pin type as Binary instead of EncodedAudio(Aac), even when the plugin source correctly declares EncodedAudio(Aac). Fix all four occurrences (processor + source macros × input + output pins): - type_discriminant: Binary → EncodedAudio - custom_type_id: also populate codec name for EncodedAudio (was only set for Custom types), so the host can round-trip the codec through the C ABI Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * style: rustfmt formatting for plugin-sdk macro Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix(moq-peer): advertise stereo channel_count for AAC in subscriber catalog The AAC-LC encoder always outputs stereo (upmixing mono input), but the subscriber catalog hardcoded channel_count=1. The client's AudioRingBuffer was initialized with 1 channel from the catalog, then received 2-channel decoded AAC frames, causing 'wrong number of channels' errors. Derive channel_count from the subscriber audio codec: AAC→2, Opus→1. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: address review findings — stale comments, dead code, cap, plugin.yml 1. Update stale API version comments in plugin-native (lib.rs, wrapper.rs) to reflect v6/v7/v8 compatibility and document that EncodedAudio is metadata-only (no runtime packet downgrade needed). 2. Remove unused CAudioCodec enum from types.rs — codec name is carried as a string via custom_type_id, not via this enum. 3. Lower FMP4_SKIP_CLASS_HARD_CAP from 100× (30,000 ≈ 10 min) to 10× (3,000 ≈ 1 min) for more reasonable memory bounds. 4. Fix plugin.yml: 'stereo' → 'mono or stereo' to match actual plugin behavior (mono input is upmixed to stereo). Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> --------- Signed-off-by: Devin AI <devin@cognition.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Signed-off-by: Devin AI <devin@streamkit.dev> Co-authored-by: StreamKit Devin <devin@streamkit.dev> Co-authored-by: Claudio Costa <cstcld91@gmail.com>
1 parent 50d3c82 commit f77825a

File tree

33 files changed

+2057
-126
lines changed

33 files changed

+2057
-126
lines changed

crates/api/src/yaml.rs

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1636,6 +1636,25 @@ nodes:
16361636
);
16371637
}
16381638

1639+
/// Regression test: `moq_aac_mixing.yml` must compile without errors.
1640+
///
1641+
/// Previously the mixer's `needs` used bracket syntax (`mic_gain[in_0]`)
1642+
/// which the parser does not support, causing a "references non-existent
1643+
/// node" error. The fix switched to simple array syntax so that
1644+
/// `Needs::Multiple` auto-generates `in_0`, `in_1` by index.
1645+
#[test]
1646+
fn test_sample_moq_aac_mixing_compiles() {
1647+
let yaml = include_str!("../../../samples/pipelines/dynamic/moq_aac_mixing.yml");
1648+
let user_pipeline = parse_yaml(yaml).unwrap();
1649+
let result = compile(user_pipeline);
1650+
1651+
assert!(
1652+
result.is_ok(),
1653+
"Sample pipeline moq_aac_mixing.yml should compile: {:?}",
1654+
result.err()
1655+
);
1656+
}
1657+
16391658
#[test]
16401659
#[allow(clippy::unwrap_used, clippy::expect_used)]
16411660
fn test_multiple_inputs_numbered_pins() {

crates/core/src/types.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ pub struct RawVideoFormat {
6363
#[non_exhaustive]
6464
pub enum AudioCodec {
6565
Opus,
66+
Aac,
6667
}
6768

6869
/// Supported encoded video codecs.

crates/engine/src/oneshot.rs

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -477,16 +477,21 @@ impl Engine {
477477
// --- 5.5. Start source / generator nodes ---
478478
// File readers need an explicit Start signal, and so do generator nodes
479479
// (e.g. video::colorbars) that follow the Ready → Start lifecycle.
480-
// In generator mode we find root nodes (never a to_node in any connection)
481-
// and send them Start as well.
480+
// We always scan for root nodes (never a to_node in any connection) so
481+
// that mixed pipelines (e.g. http_input + colorbars) work correctly.
482+
// http_input nodes are excluded because they are driven by the incoming
483+
// HTTP stream rather than a Start signal.
482484
let mut start_node_ids: Vec<String> = source_node_ids.clone();
483485

484-
if !has_http_input && source_node_ids.is_empty() {
485-
// Generator mode — find root nodes that need a Start signal.
486+
{
486487
let downstream_nodes: std::collections::HashSet<&str> =
487488
definition.connections.iter().map(|c| c.to_node.as_str()).collect();
488489
for name in definition.nodes.keys() {
489-
if name != &output_node_id && !downstream_nodes.contains(name.as_str()) {
490+
if name != &output_node_id
491+
&& !downstream_nodes.contains(name.as_str())
492+
&& !start_node_ids.contains(name)
493+
&& !http_input_nodes.contains(name)
494+
{
490495
start_node_ids.push(name.clone());
491496
}
492497
}

crates/nodes/src/containers/mp4.rs

Lines changed: 255 additions & 42 deletions
Large diffs are not rendered by default.

crates/nodes/src/transport/moq/constants.rs

Lines changed: 68 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,22 @@ use streamkit_core::types::{
1212
AudioCodec, EncodedAudioFormat, EncodedVideoFormat, PacketMetadata, PacketType, VideoCodec,
1313
};
1414

15-
pub const DEFAULT_AUDIO_FRAME_DURATION_US: u64 = 20_000;
15+
pub const DEFAULT_AUDIO_FRAME_DURATION_US_OPUS: u64 = 20_000;
16+
pub const DEFAULT_AUDIO_FRAME_DURATION_US_AAC: u64 = 21_333;
17+
18+
/// Return the default audio frame duration for the given codec.
19+
pub const fn default_audio_frame_duration_us(codec: AudioCodec) -> u64 {
20+
match codec {
21+
AudioCodec::Aac => DEFAULT_AUDIO_FRAME_DURATION_US_AAC,
22+
_ => DEFAULT_AUDIO_FRAME_DURATION_US_OPUS,
23+
}
24+
}
1625

1726
pub fn packet_duration_us(metadata: Option<&PacketMetadata>) -> Option<u64> {
1827
metadata.and_then(|m| m.duration_us).filter(|d| *d > 0)
1928
}
2029

21-
/// Return the accepted media types for dynamic MoQ pins (Opus audio + VP9/AV1/H264 video).
30+
/// Return the accepted media types for dynamic MoQ pins (Opus/AAC audio + VP9/AV1/H264 video).
2231
///
2332
/// This is shared across `moq_peer` and `moq_push` to avoid duplicating the
2433
/// type construction in every `RequestAddInputPin` / `RequestAddOutputPin` handler.
@@ -28,6 +37,10 @@ pub fn moq_accepted_media_types() -> Vec<PacketType> {
2837
codec: AudioCodec::Opus,
2938
codec_private: None,
3039
}),
40+
PacketType::EncodedAudio(EncodedAudioFormat {
41+
codec: AudioCodec::Aac,
42+
codec_private: None,
43+
}),
3144
PacketType::EncodedVideo(EncodedVideoFormat {
3245
codec: VideoCodec::Vp9,
3346
bitstream_format: None,
@@ -132,6 +145,59 @@ pub fn parse_video_codec_config(s: &str) -> Option<VideoCodec> {
132145
}
133146
}
134147

148+
/// Build the [`hang::catalog::AudioCodec`] entry for a given [`AudioCodec`].
149+
///
150+
/// Centralises the AAC/Opus catalog construction, mirroring
151+
/// [`catalog_video_codec`] for the video side.
152+
pub fn catalog_audio_codec(codec: AudioCodec) -> hang::catalog::AudioCodec {
153+
match codec {
154+
AudioCodec::Opus => hang::catalog::AudioCodec::Opus,
155+
AudioCodec::Aac => hang::catalog::AudioCodec::AAC(hang::catalog::AAC { profile: 2 }),
156+
// Future-proof: fall back to Opus and warn.
157+
_ => {
158+
tracing::warn!(?codec, "unsupported AudioCodec for MoQ catalog, defaulting to Opus");
159+
hang::catalog::AudioCodec::Opus
160+
},
161+
}
162+
}
163+
164+
/// Resolve the audio codec from config → input_types → default (Opus).
165+
///
166+
/// Priority order:
167+
/// 1. Explicit `audio_codec` config param (required for dynamic pipelines)
168+
/// 2. Auto-detected from `input_types` (static pipelines)
169+
/// 3. Default: Opus
170+
///
171+
/// Shared by `moq_peer` and `moq_push` to avoid duplicating the resolution chain.
172+
pub fn resolve_audio_codec(
173+
config_codec: Option<&str>,
174+
input_types: &std::collections::HashMap<String, PacketType>,
175+
) -> AudioCodec {
176+
config_codec
177+
.and_then(parse_audio_codec_config)
178+
.or_else(|| {
179+
input_types.iter().find_map(|(_, pt)| match pt {
180+
PacketType::EncodedAudio(fmt) => Some(fmt.codec),
181+
_ => None,
182+
})
183+
})
184+
.unwrap_or(AudioCodec::Opus)
185+
}
186+
187+
/// Parse an `audio_codec` config string (e.g. `"opus"`, `"aac"`) into the
188+
/// corresponding [`AudioCodec`]. Returns `None` for unrecognised values so
189+
/// the caller can fall back to auto-detection.
190+
pub fn parse_audio_codec_config(s: &str) -> Option<AudioCodec> {
191+
match s.to_ascii_lowercase().as_str() {
192+
"opus" => Some(AudioCodec::Opus),
193+
"aac" => Some(AudioCodec::Aac),
194+
_ => {
195+
tracing::warn!(audio_codec = %s, "unrecognised audio_codec config value — ignoring");
196+
None
197+
},
198+
}
199+
}
200+
135201
#[cfg(test)]
136202
mod tests {
137203
use super::*;

crates/nodes/src/transport/moq/peer/config.rs

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ use serde::Deserialize;
1111
use std::collections::HashMap;
1212
use std::sync::Arc;
1313
use streamkit_core::timing::MediaClock;
14-
use streamkit_core::types::{Packet, PacketType, VideoCodec};
14+
use streamkit_core::types::{AudioCodec, Packet, PacketType, VideoCodec};
1515
use tokio::sync::{broadcast, mpsc, watch, Semaphore};
1616

1717
// ── Internal helper types ────────────────────────────────────────────────────
@@ -89,6 +89,16 @@ pub(super) enum PublisherEvent {
8989
Disconnected { path: String, error: Option<String> },
9090
}
9191

92+
/// Resolved codec pair for video and audio.
93+
///
94+
/// Bundles the two codec fields that would otherwise be passed as separate
95+
/// parameters to `handle_pin_management` and friends.
96+
#[derive(Debug, Clone, Copy)]
97+
pub(super) struct MediaCodecConfig {
98+
pub video: VideoCodec,
99+
pub audio: AudioCodec,
100+
}
101+
92102
/// Media and output configuration shared across subscriber-related functions.
93103
pub(super) struct SubscriberMediaConfig {
94104
pub has_video: bool,
@@ -98,6 +108,7 @@ pub(super) struct SubscriberMediaConfig {
98108
pub output_group_duration_ms: u64,
99109
pub output_initial_delay_ms: u64,
100110
pub video_codec: VideoCodec,
111+
pub audio_codec: AudioCodec,
101112
}
102113

103114
pub(super) struct BidirectionalTaskConfig {
@@ -157,6 +168,8 @@ pub(super) struct SubscriberSendCtx<'a> {
157168
pub last_audio_ts_ms: Option<u64>,
158169
pub last_video_ts_ms: Option<u64>,
159170
pub stats_delta_tx: &'a mpsc::Sender<NodeStatsDelta>,
171+
/// Resolved audio codec — used for codec-aware default frame durations.
172+
pub audio_codec: AudioCodec,
160173
}
161174

162175
// ── Shared map of dynamic output pin senders ─────────────────────────────────
@@ -313,6 +326,31 @@ pub struct MoqPeerConfig {
313326
/// is auto-detected from `input_types` (static pipelines) and falls back
314327
/// to VP9.
315328
pub video_codec: Option<String>,
329+
/// Audio codec for the MoQ catalog.
330+
///
331+
/// Required for dynamic pipelines where `input_types` is not available at
332+
/// startup. Accepted values: `"opus"`, `"aac"`. When `None`, the codec
333+
/// is auto-detected from `input_types` (static pipelines) and falls back
334+
/// to Opus.
335+
///
336+
/// Controls the **publisher output pin** type (`audio/data`). For
337+
/// transcoding scenarios where the subscriber receives a different codec
338+
/// (e.g. Opus in → AAC out), use [`subscriber_audio_codec`] to override
339+
/// the subscriber catalog codec independently.
340+
pub audio_codec: Option<String>,
341+
/// Audio codec advertised in the **subscriber** MoQ catalog.
342+
///
343+
/// When set, overrides [`audio_codec`] for the subscriber side only
344+
/// (catalog, frame duration). The publisher output pin (`audio/data`)
345+
/// continues to use [`audio_codec`].
346+
///
347+
/// Useful for transcoding pipelines where the publisher sends one codec
348+
/// (e.g. Opus) but the pipeline re-encodes to another (e.g. AAC) before
349+
/// feeding it back to subscribers.
350+
///
351+
/// Accepted values: `"opus"`, `"aac"`. When `None`, falls back to
352+
/// [`audio_codec`].
353+
pub subscriber_audio_codec: Option<String>,
316354
}
317355

318356
impl Default for MoqPeerConfig {
@@ -327,6 +365,8 @@ impl Default for MoqPeerConfig {
327365
video_width: 640,
328366
video_height: 480,
329367
video_codec: None,
368+
audio_codec: None,
369+
subscriber_audio_codec: None,
330370
}
331371
}
332372
}

0 commit comments

Comments
 (0)