feat(plugin-sdk): add frame pool allocation for native plugins#249
Merged
streamer45 merged 5 commits intomainfrom Apr 5, 2026
Merged
feat(plugin-sdk): add frame pool allocation for native plugins#249streamer45 merged 5 commits intomainfrom
streamer45 merged 5 commits intomainfrom
Conversation
Implement zero-copy buffer semantics for native plugin video/audio frames via host-side frame pool allocation, eliminating per-frame heap allocations from .to_vec() copies in the conversions layer. Changes: - Bump NATIVE_PLUGIN_API_VERSION from 5 to 6 - Add CAllocResult, CAllocAudioResult, CAllocVideoFn, CAllocAudioFn types - Consolidate callback parameters into CNodeCallbacks struct - Extend CVideoFrame/CAudioFrame with buffer_handle and metadata fields - Add PooledVideoBuffer/PooledAudioBuffer SDK wrappers with linear-type semantics (Drop returns buffer to pool if not consumed) - Add OutputSender::alloc_video/alloc_audio/send_video/send_audio methods - Implement host-side alloc/free shims in wrapper.rs using frame pools - Update native_plugin_entry! and native_source_plugin_entry! macros - Update Slint plugin to use pool allocation with legacy fallback - Extend CPacketOwned with VideoOwned/AudioOwned for metadata ownership DHAT profiling (300-frame Slint watermark pipeline): Before: 1,042,800,741 bytes total, 9,504,000 bytes from to_vec copies After: 252,676,535 bytes total, 0 bytes from to_vec copies Result: 75.8% total allocation reduction, to_vec copies eliminated Signed-off-by: Devin AI <devin@cognition.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
Signed-off-by: Devin AI <devin@cognition.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Critical fixes: - Free pooled buffer_handle in packet_from_c when data/samples pointer is null (prevents leak on malformed frames) - Free pooled buffer_handle in output_callback_shim when pin name parsing fails before packet_from_c can reclaim it API improvements: - Remove redundant data_len from send_video (use buf.len() instead) - Remove redundant sample_count from send_audio (use buf.sample_count()) - Add struct_size field to CNodeCallbacks for forward compatibility - Extract duplicated CResult-to-Result pattern into result_from_c helper Naming/exports: - Rename CAllocResult to CAllocVideoResult for consistency with CAllocAudioResult - Re-export PooledVideoBuffer and PooledAudioBuffer in the prelude Tests: - Add regression tests for buffer_handle cleanup on null data/samples error paths in packet_from_c Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
Check free_fn.is_none() alongside data.is_null() before constructing PooledVideoBuffer/PooledAudioBuffer. A buggy host returning non-null data with free_fn: None would previously leak via the ? operator silently returning None without cleanup. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
- Remove const qualifier from as_mut_slice/consume on PooledVideoBuffer and PooledAudioBuffer — these dereference heap pointers and will never be called in const context. Add #[allow(clippy::missing_const_for_fn)] with rationale comment. - Downgrade info! to trace! in __plugin_flush macro — these are development debugging lines that would spam production logs. Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Zero-copy buffer semantics for native plugin video/audio frames via host-side frame pool allocation. Eliminates per-frame
memcpyon the hot path by letting plugins write directly into pool-allocated buffers and pass them back by handle.DHAT profiling results (300-frame Slint watermark pipeline):
to_veccopy site (9,504,000 bytes / 300 blocks): eliminated entirelyChanges
SDK (
sdks/plugin-sdk/native/src/)PooledVideoBuffer/PooledAudioBufferlinear-type wrappers withDrop-based pool returnOutputSender::alloc_video(),alloc_audio(),send_video(),send_audio()for zero-copy pathCNodeCallbacksstruct consolidating output + telemetry + alloc callbacks (replaces positional params)CNodeCallbacks.struct_sizefield for forward-compatible extensibilityresult_from_c()helper extracting duplicated CResult→Result conversionCAllocVideoResult/CAllocAudioResultallocation result typesHost (
crates/plugin-native/src/wrapper.rs)CallbackContextextended withVideoFramePool/AudioFramePoolalloc_video_shim/alloc_audio_shim/free_*_bufferC callbacksfree_packet_buffer_handle()safety net: frees pooled handles on callback error pathsbuild_node_callbacks()constructs the consolidated struct withstruct_sizeConversions (
sdks/plugin-sdk/native/src/conversions.rs)packet_from_c: zero-copy branch reclaimsPooledVideoData/PooledSamplesviaBox::from_rawpacket_from_c: null-data/null-samples error paths now freebuffer_handlebefore returning (fixes buffer leak)packet_to_c: extendedCPacketOwned::Video/Audiowith metadata storage (prevents dangling pointer)Slint plugin
Review feedback addressed
From reviewer:
packet_from_cwhen data/samples is null butbuffer_handleis non-nulloutput_callback_shimwhen pin name parsing fails beforepacket_from_crunsdata_len/sample_countparams fromsend_video/send_audio— now usesbuf.len()/buf.sample_count()const fnfromas_mut_slice/consume— these dereference heap pointers and will never be called in const context. Added#[allow(clippy::missing_const_for_fn)]with rationale.struct_sizetoCNodeCallbacksfor v7-on-v6 forward compatresult_from_c()helper for CResult→Result conversion (4 call sites)CAllocResult→CAllocVideoResultPooledVideoBuffer/PooledAudioBufferto prelude!Send: Acknowledged — both buffer types are!Senddue to raw pointers, consistent withOutputSender. If cross-thread buffer passing is needed later, explicitunsafe impl Sendwould be required.From Devin Review:
alloc_video/alloc_audiowhenfree_fnisNonebut allocation succeeded — now checksfree_fn.is_none()alongsidedata.is_null()Additional:
info!→trace!for debug logging in__plugin_flushmacro (4 lines that would spam production logs)Review & Testing Checklist for Human
Box::into_raw→Box::from_rawround-trip inalloc_video_shim→packet_from_cis sound (theVecheap pointer survives theBoxindirection)free_packet_buffer_handleinoutput_callback_shimcorrectly handles all error paths wherebuf.consume()has already suppressedDropstruct_sizefield: VerifyCNodeCallbacksstruct layout with the new leadingstruct_size: usizefield doesn't break ABI alignment for any callback pointer fieldsalloc_videoreturning null (no pool) — should fall through to legacy.to_vec()copy pathNotes
CNodeCallbacksconsolidation is handled transparently by the macros.Link to Devin session: https://staging.itsdev.in/sessions/ed6a1517580e43839384cd56691bf505
Requested by: @streamer45