Commit 169dd9a
fix(compositor): improve image overlay quality, caching, aspect ratio, and selectability (#78)
* fix(compositor): improve image overlay quality, caching, aspect ratio, and selectability
- Replace nearest-neighbor prescaling with bilinear (image crate Triangle
filter) for much better rendering of images containing text or fine detail
- Cache decoded image overlays across UpdateParams calls — only re-decode
when data_base64 or target rect dimensions change, reusing existing
Arc<DecodedOverlay> otherwise
- Lock aspect ratio for image layers during resize (same as video layers)
- Show actual image thumbnail in compositor canvas UI for easier selection;
switch border from dotted to solid, remove crosshatch pattern
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): guard against index mismatch in image overlay cache
Use old_imgs.get(i) instead of old_imgs[i] to avoid a panic when
a previous decode_image_overlay call failed, leaving old_imgs shorter
than old_cfgs.
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): address review — proper index mapping for cache, broader MIME detection
- Build a HashMap<usize, &Arc<DecodedOverlay>> by walking old configs and
decoded overlays in tandem, so cache lookups use config index rather than
assuming positional alignment (which breaks when a previous decode failed)
- Add WebP and GIF magic-byte detection for image thumbnail data URIs
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* style(compositor): apply cargo fmt formatting
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): fix HashMap type and double-deref in overlay cache
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): content-keyed overlay cache with dimension-based matching
Replace incorrect positional index mapping with a content-keyed cache
that matches decoded overlays to configs by comparing prescaled bitmap
dimensions against the config's target rect. This correctly handles
the case where a mid-list decode failure makes the decoded slice shorter
than the config vec — failed configs are skipped (not consumed) because
their target dimensions won't match the next decoded overlay.
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): default image overlay z-index to 200 so it renders above video layers
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* style(compositor): add rationale comment for clippy::expect_used suppression
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* style: apply formatting fixes (cargo fmt + prettier)
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): address review findings #1-#4, #7
- Replace dimension-matching cache heuristic with index-based mapping
using image_overlay_cfg_indices (finding #1)
- Only update x/y position on cache hit, not full rect clone (finding #2)
- Fix MIME sniffing comment wording to 'base64-encoded magic bytes',
add BMP detection (finding #3)
- Switch from data-URI to URL.createObjectURL with cleanup for image
overlay thumbnails (finding #4)
- Change SAFETY comment to Invariant in prescale_rgba (finding #7)
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): preserve image aspect ratio, add image layer controls, optimize base64 decode
- Backend: prescale images with aspect-ratio preservation (scale-to-fit
instead of stretch-to-fill) and centre within the target rect.
- Backend: re-centre cached overlays on position update.
- Frontend: detect natural image dimensions on add and set initial rect
to match source aspect ratio.
- Frontend: add opacity/rotation slider controls for selected image
overlays (matching video and text layer controls).
- Frontend: fix findAnyLayer to pass through rotationDegrees and zIndex
for image overlays instead of hardcoding 0.
- Frontend: replace O(n) atob + byte-by-byte loop with fetch(data-URI)
for more efficient base64-to-blob conversion.
- Frontend: remove BMP MIME detection (inconsistent browser support).
- Frontend: add z-index band allocation comments (video 0-99, text
100-199, image 200+).
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): apply rotation transform to image overlay layer in canvas preview
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
* fix(compositor): include rotationDegrees and zIndex in overlay sync change detection
Add rotationDegrees and zIndex to the image overlay change-detection
comparisons in the params sync effect so that YAML or backend changes
to these fields are reflected in the UI. Also add the missing zIndex
check to the text overlay change detection for consistency.
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
---------
Co-authored-by: StreamKit Devin <devin@streamkit.dev>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>1 parent 787e0cb commit 169dd9a
File tree
5 files changed
+300
-87
lines changed- crates/nodes/src/video/compositor
- ui/src
- components
- hooks
- nodes
5 files changed
+300
-87
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| |||
252 | 253 | | |
253 | 254 | | |
254 | 255 | | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
255 | 262 | | |
256 | 263 | | |
| 264 | + | |
| 265 | + | |
257 | 266 | | |
258 | 267 | | |
259 | 268 | | |
| |||
268 | 277 | | |
269 | 278 | | |
270 | 279 | | |
| 280 | + | |
271 | 281 | | |
272 | 282 | | |
273 | 283 | | |
| |||
408 | 418 | | |
409 | 419 | | |
410 | 420 | | |
| 421 | + | |
411 | 422 | | |
412 | 423 | | |
413 | 424 | | |
| |||
489 | 500 | | |
490 | 501 | | |
491 | 502 | | |
| 503 | + | |
492 | 504 | | |
493 | 505 | | |
494 | 506 | | |
| |||
529 | 541 | | |
530 | 542 | | |
531 | 543 | | |
| 544 | + | |
532 | 545 | | |
533 | 546 | | |
534 | 547 | | |
| |||
651 | 664 | | |
652 | 665 | | |
653 | 666 | | |
| 667 | + | |
654 | 668 | | |
655 | 669 | | |
656 | 670 | | |
| |||
666 | 680 | | |
667 | 681 | | |
668 | 682 | | |
669 | | - | |
670 | | - | |
671 | | - | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
672 | 720 | | |
673 | | - | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
674 | 749 | | |
675 | | - | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
676 | 754 | | |
677 | 755 | | |
678 | 756 | | |
679 | 757 | | |
| 758 | + | |
680 | 759 | | |
681 | 760 | | |
682 | 761 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
54 | 54 | | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
55 | 76 | | |
56 | | - | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
57 | 88 | | |
58 | 89 | | |
59 | | - | |
60 | | - | |
61 | | - | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
62 | 93 | | |
63 | 94 | | |
64 | 95 | | |
| |||
76 | 107 | | |
77 | 108 | | |
78 | 109 | | |
79 | | - | |
80 | | - | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
81 | 115 | | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
97 | 123 | | |
98 | 124 | | |
99 | 125 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | 114 | | |
127 | 115 | | |
128 | 116 | | |
| |||
201 | 189 | | |
202 | 190 | | |
203 | 191 | | |
204 | | - | |
205 | | - | |
206 | | - | |
207 | | - | |
208 | | - | |
209 | | - | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
217 | | - | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | | - | |
222 | | - | |
223 | | - | |
224 | 192 | | |
225 | 193 | | |
226 | 194 | | |
| |||
437 | 405 | | |
438 | 406 | | |
439 | 407 | | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
440 | 448 | | |
441 | 449 | | |
442 | 450 | | |
| |||
447 | 455 | | |
448 | 456 | | |
449 | 457 | | |
| 458 | + | |
| 459 | + | |
450 | 460 | | |
451 | | - | |
| 461 | + | |
452 | 462 | | |
453 | 463 | | |
454 | | - | |
455 | | - | |
456 | 464 | | |
457 | 465 | | |
458 | 466 | | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
459 | 482 | | |
460 | | - | |
461 | | - | |
462 | | - | |
463 | 483 | | |
464 | 484 | | |
465 | 485 | | |
| |||
0 commit comments