Commit c7f1ddd
feat(transport): add dynamic pin support to moq_peer and moq_push (#200)
* feat(transport): add dynamic pin support to moq_peer and moq_push
Generalize MoQ transport nodes to discover and create tracks/pins
dynamically from catalogs instead of hardcoding audio+video pairs.
moq_peer changes:
- Set supports_dynamic_pins() to true
- Thread DynamicOutputs (Arc<RwLock<HashMap>>) through the publisher
call chain: run -> start_publisher_task_with_permit ->
publisher_receive_loop -> watch_catalog_and_process ->
spawn_track_processor -> process_publisher_frames ->
process_frame_from_group
- In watch_catalog_and_process, build track-named dynamic pin names
(e.g. audio/data, video/hd) from catalog entries
- In process_frame_from_group, send frames to both the dynamic
(track-named) output pin and the legacy pin for backward compat
- Handle all PinManagementMessage variants in handle_pin_management
- Accept both EncodedAudio(Opus) and EncodedVideo(VP9) on both
input pins (in/in_1) for flexible media routing
moq_push changes:
- Set supports_dynamic_pins() to true
- Accept both EncodedAudio(Opus) and EncodedVideo(VP9) on both
input pins (in/in_1)
- Handle dynamic input pin creation via PinManagementMessage,
mapping each new pin to a corresponding MoQ track
- Add pin management select branch in the run loop
Engine changes (dynamic_actor.rs):
- In validate_connection_types, skip strict type validation for
source pins on nodes that support dynamic pins
- In connect_nodes, create output pins on-demand via
RequestAddOutputPin -> AddedOutputPin flow when the pin
distributor doesn't exist but the node supports dynamic pins
All existing pipeline YAML files continue to work unchanged.
Legacy out/out_1 and in/in_1 pins remain as stable fallbacks.
Refs: #197
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): address review feedback on dynamic pin support
- Poll dynamic input receivers in moq_push select loop using poll_fn
- Determine is_video from pin name prefix convention instead of accepts_types
- Forward dynamic input pin packets in moq_peer instead of dropping channel
- Use DynamicInputState struct instead of tuple for type clarity
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* test(transport): add regression tests for dynamic pin fixes
- Test make_dynamic_output_pin produces correct types for video/audio/bare names
- Test AddedInputPin channel is not dropped (regression for channel discard bug)
- Test is_video determination uses pin name prefix convention
- Test track name derivation from pin names
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): fix double-prefixed pin names and shutdown cleanup
- Use catalog track names directly (already prefixed) instead of
re-prefixing with audio/ or video/, which caused double-prefixed
names like 'audio/audio/data'
- Finish dynamic input track producers on MoqPushNode shutdown
- Add regression test for double-prefix bug
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): finish track producers on remove, add stats to dynamic input forwarding
- RemoveInputPin now calls finish() on track producers before dropping
- Dynamic input forwarding tasks in moq_peer report received/sent stats
via stats_delta_tx, matching the static pin handler pattern
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* refactor(transport): remove legacy pin names, use track-named pins exclusively
BREAKING CHANGE: moq_peer output pins renamed from out/out_1 to
audio/data and video/data to match catalog track names. Removes
audio_output_pin/video_output_pin parameters from the entire publisher
call chain (start_publisher_task_with_permit, publisher_receive_loop,
watch_catalog_and_process, spawn_track_processor, process_publisher_frames,
process_frame_from_group). Unifies output_pin and dynamic_pin_name into
a single track-name-based output pin. Updates all sample pipeline YAML
files to reference the new pin names.
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix: update remaining out_1 references in samples, e2e fixtures, tests, and docs
Updates missed references to the old moq_peer out/out_1 pin names:
- samples/pipelines/dynamic/video_moq_webcam_pip.yml
- samples/pipelines/dynamic/video_moq_screen_share.yml
- e2e/fixtures/webcam-pip.yaml
- e2e/fixtures/webcam-pip-cropped.yaml
- e2e/fixtures/webcam-pip-circle.yaml
- crates/api/src/yaml.rs (parser tests)
- docs/src/content/docs/guides/creating-pipelines.md
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* refactor(transport): address all 9 review items on dynamic pin support
Critical fixes:
- Exclusive routing: dynamic channel OR static output, never both (fix #1)
- RwLock poison logged as error instead of silently swallowed (fix #2)
Improvements:
- Spawned input-forwarding task uses tokio::select! with shutdown_rx (fix #3)
- validate_connection_types logs at warn for dynamic pin skip (fix #4)
- Document poll_fn starvation bias as accepted trade-off (fix #5)
- Remove unused channels parameter from handle_pin_management (fix #6)
Nits:
- Update DynamicOutputs doc comment, remove stale legacy reference (fix #7)
- Use Arc short form (already imported) (fix #8)
- Improve test to exercise MoqPeerNode::new + output_pins + make_dynamic_output_pin (fix #9)
Also refactored handle_pin_management and process_frame_from_group to
reduce cognitive complexity below the 50-point lint threshold by
extracting route_packet, spawn_dynamic_input_forwarder,
insert_dynamic_output, remove_dynamic_output, and make_dynamic_input_pin
helper methods.
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): eliminate TOCTOU race in route_packet
Hold a single read lock for both the existence check and the send in
route_packet, preventing a concurrent RemoveOutputPin from removing the
entry between two separate lock acquisitions which would silently drop
the packet.
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): handle closed dynamic output channels in route_packet
Distinguish try_send results: Ok and Full return true (packet sent or
acceptable frame drop for real-time media), Closed returns false to
trigger shutdown — matching the static output path behaviour.
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): keep track processor alive on closed dynamic channel
A closed dynamic output channel (downstream consumer disconnected)
now removes the stale entry and continues instead of triggering
FrameResult::Shutdown. This prevents a single consumer disconnect
from killing the entire track processor.
Also extract track_name_from_pin() and is_video_pin() into named
functions in push.rs so tests exercise the real production code
instead of duplicating the logic inline.
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): keep dynamic input forwarder alive when no subscribers
Match the static input path behaviour: discard frames with
`let _ = tx.send(frame)` instead of breaking out of the loop when
there are no active broadcast receivers. This prevents the dynamic
input forwarder from permanently shutting down between subscriber
connections.
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): address review round 3 — catalog republish, single-lock route_packet, cleanup
- Re-publish MoQ catalog when dynamic tracks are added/removed (push.rs)
- Merge route_packet double RwLock acquisition into single lock with RouteOutcome enum
- Add design rationale comment on std::sync::RwLock choice for DynamicOutputs
- Extract moq_accepted_media_types() helper, deduplicate across peer/mod.rs and push.rs
- Change dynamic pin validation log from warn to debug (dynamic_actor.rs)
- Use Arc::default() consistently for DynamicOutputs construction
- Update moq_peer.yml comment to mention video/data output pin
- Remove unused type imports from push.rs
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(transport): downgrade catalog republish log to debug
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix: address round 4 review — packet drops, forwarder lifecycle, type validation
- route_packet: match on TrySendError::Full/Closed via RouteOutcome enum,
log dropped packets at debug level instead of silently discarding
- Store JoinHandle for each dynamic input forwarder in a HashMap;
abort on RemoveInputPin to prevent task leaks
- After dynamic output pin creation in connect_nodes, validate type
compatibility using can_connect_any before wiring
- republish_catalog returns bool; on failure roll back catalog entry
and skip adding DynamicInputState
- Use swap_remove instead of remove for O(1) dynamic_inputs removal
- Consistent lock-poisoning recovery via unwrap_or_else(PoisonError::into_inner)
- Align default dynamic pin names (in_dyn → dynamic_in)
- Extract activate_dynamic_input, insert/remove_catalog_rendition helpers
to stay within cognitive_complexity limit
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix: clean up stale resources on dynamic pin creation failures
- Type-mismatch early return in connect_nodes now removes the orphaned
PinDistributor entry and stale pin metadata before returning
- AddedOutputPin send failure path gets the same cleanup
- Document that validate_connection_types skips dest-pin validation too
when source node supports dynamic pins (known limitation)
- RemoveInputPin in push.rs uses swap_remove instead of drain+collect
- Prune finished forwarder JoinHandles on AddedInputPin to prevent
unbounded growth from naturally-closed channels
- Add safety comment about poll_fn/select! mutable borrow interaction
- Deduplicate output_pins() by reusing make_dynamic_output_pin
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix: finish leaked producers, shut down orphaned distributors, cleanup nits
- activate_dynamic_input: finish track producer before returning on
catalog republish failure to avoid dangling broadcast track
- connect_nodes: send PinConfigMsg::Shutdown to the spawned
PinDistributor on both type-mismatch and AddedOutputPin send failure
error paths, preventing orphaned actor tasks
- Abort all forwarder JoinHandles on node shutdown for deterministic
cleanup instead of relying on channel close propagation
- Remove redundant 'let mut catalog_producer = catalog_producer' rebinding
- Downgrade subscriber_count atomics from SeqCst to Relaxed (only used
for logging, no cross-variable synchronization needed)
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix: rollback leaked input pins, guard duplicates, timeout pin creation
- Add rollback_dynamic_input helper to clean up destination input pins
when step-2 (output pin creation) fails in connect_nodes
- Track created_dynamic_input to conditionally rollback on all 6 step-2
failure paths (type mismatch, send failures, timeouts)
- Wrap RequestAddInputPin and RequestAddOutputPin responses with
tokio::time::timeout(5s) to prevent engine deadlock
- Guard duplicate dynamic input pin names in push.rs with
check-and-replace via swap_remove
- Abort old forwarder handle on re-add collision in peer/mod.rs
- Extract activate_dynamic_input_forwarder to reduce cognitive complexity
- Bump stale dynamic output entry log from debug to info
- Make original catalog binding mut, remove redundant rebind
- Align moq_accepted_media_types() import qualification
- Shut down orphaned PinDistributor actors on type-mismatch and
AddedOutputPin send failure paths
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix: assert pin.name == from_pin invariant on dynamic output creation
Add debug_assert_eq! after receiving the pin definition from
RequestAddOutputPin to make the implicit contract explicit: the
node must return the suggested name unchanged.
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
---------
Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-authored-by: StreamKit Devin <devin@streamkit.dev>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>1 parent d739172 commit c7f1ddd
File tree
21 files changed
+1147
-161
lines changed- crates
- api/src
- engine/src
- nodes/src/transport/moq
- peer
- docs/src/content/docs/guides
- e2e/fixtures
- samples/pipelines/dynamic
21 files changed
+1147
-161
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1145 | 1145 | | |
1146 | 1146 | | |
1147 | 1147 | | |
1148 | | - | |
| 1148 | + | |
| 1149 | + | |
1149 | 1150 | | |
1150 | 1151 | | |
1151 | 1152 | | |
| |||
1176 | 1177 | | |
1177 | 1178 | | |
1178 | 1179 | | |
1179 | | - | |
| 1180 | + | |
| 1181 | + | |
1180 | 1182 | | |
1181 | 1183 | | |
1182 | 1184 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
667 | 667 | | |
668 | 668 | | |
669 | 669 | | |
670 | | - | |
671 | | - | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
672 | 690 | | |
673 | 691 | | |
674 | 692 | | |
| |||
777 | 795 | | |
778 | 796 | | |
779 | 797 | | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
780 | 801 | | |
781 | 802 | | |
782 | 803 | | |
| |||
802 | 823 | | |
803 | 824 | | |
804 | 825 | | |
805 | | - | |
806 | | - | |
807 | | - | |
808 | | - | |
809 | | - | |
810 | | - | |
811 | | - | |
812 | | - | |
813 | | - | |
814 | | - | |
815 | | - | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
816 | 851 | | |
817 | 852 | | |
818 | 853 | | |
| |||
841 | 876 | | |
842 | 877 | | |
843 | 878 | | |
| 879 | + | |
844 | 880 | | |
845 | 881 | | |
846 | 882 | | |
| |||
852 | 888 | | |
853 | 889 | | |
854 | 890 | | |
855 | | - | |
856 | | - | |
857 | | - | |
858 | | - | |
859 | | - | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
860 | 900 | | |
861 | 901 | | |
862 | 902 | | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
| 991 | + | |
| 992 | + | |
| 993 | + | |
| 994 | + | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
| 1001 | + | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
| 1009 | + | |
| 1010 | + | |
| 1011 | + | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
| 1052 | + | |
| 1053 | + | |
| 1054 | + | |
| 1055 | + | |
| 1056 | + | |
| 1057 | + | |
| 1058 | + | |
| 1059 | + | |
| 1060 | + | |
| 1061 | + | |
863 | 1062 | | |
864 | 1063 | | |
865 | 1064 | | |
| |||
881 | 1080 | | |
882 | 1081 | | |
883 | 1082 | | |
| 1083 | + | |
| 1084 | + | |
| 1085 | + | |
| 1086 | + | |
| 1087 | + | |
| 1088 | + | |
| 1089 | + | |
| 1090 | + | |
| 1091 | + | |
| 1092 | + | |
| 1093 | + | |
| 1094 | + | |
| 1095 | + | |
| 1096 | + | |
| 1097 | + | |
| 1098 | + | |
| 1099 | + | |
| 1100 | + | |
| 1101 | + | |
884 | 1102 | | |
885 | 1103 | | |
886 | 1104 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
8 | 10 | | |
9 | 11 | | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
13 | 15 | | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
0 commit comments