Skip to content

Commit b8d9051

Browse files
staging-devin-ai-integration[bot]streamkit-devinstreamer45
authored
feat(compositor): text color, font selection, draggable layers, clipping fix (#80)
* feat(compositor): text color, font selection, draggable layers, clipping fix - Text color: add color picker (RGB + alpha slider) for text overlays - Font selection: add font_name field to TextOverlayConfig with 12 curated system fonts (DejaVu, Liberation, FreeFonts), dropdown in UI, warning when named font file is missing - Draggable layer list: replace z-index ▲/▼ buttons with drag-to-reorder using motion/react Reorder, reassigns z-indices on drop - Text clipping fix: expand bitmap height to ceil(font_size * 1.4) in rasterize_text_overlay and auto-expand UI rect height when font size increases Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * style: apply rustfmt to overlay.rs Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: batch reorder z-index updates and add color comparison - Add reorderLayers() to useCompositorLayers that atomically updates z-index for all layer types (video, text, image) in a single commit, preventing race conditions from stale refs when handleReorder fired N individual update calls. - Add missing color array comparison to mergeOverlayState's hasExtraChanges comparator so server-echoed color changes are correctly detected. - Remove unused onZIndexChange prop from UnifiedLayerList since reorderLayers now handles all z-index mutations. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: use dynamic max z-index for new overlays instead of fixed bands Replace fixed z-index bands (text: 100+, image: 200+) with maxZIndex() + 1 so that new overlays always stack on top even after drag-to-reorder has normalized z-indices to [0, n-1]. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: expand text overlay bitmap to measured text dimensions Add measure_text() that computes actual pixel width/height from font metrics, and use it in rasterize_text_overlay to expand the bitmap to fit the full rendered string. Previously only height was expanded via a 1.4× heuristic; now both width and height use exact font measurements. On the UI side, updateTextOverlay now auto-expands the rect width (~0.6× font_size per character) in addition to height when the text would overflow the current box. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * style: cargo fmt Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: address review — atomic reorder commit and complete font list - Replace throttledConfigChange with immediate onParamChange in reorderLayers' onParamChange path so video layer and overlay z-index updates commit atomically in the same tick. - Extract serializeLayers() helper to avoid duplicating layer serialization logic between buildConfig and reorderLayers. - Add missing dejavu-serif-bold and dejavu-sans-mono-bold to the font_name doc comment in TextOverlayConfig. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * feat: render font selection in canvas preview and fix dropdown contrast - Add FONT_FAMILY_MAP to CompositorCanvas that maps backend font names to CSS font-family values. Text overlays now preview the selected font in both display and edit mode on the canvas. - Bold font variants (e.g. dejavu-sans-bold) render with fontWeight 700. - Fix FontSelect contrast: use var(--sk-panel-bg) instead of undefined var(--sk-input-bg), add color-scheme hint, and style option elements explicitly for dark mode compatibility. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: improve font preview fallbacks and remove alpha slider - Add web-safe intermediate fonts (Verdana, Georgia, Arial, Times New Roman, Courier New) to the CSS font-family fallback stacks so the canvas preview shows a visible difference between sans-serif, serif, and monospace font groups even when the exact system fonts are not installed in the browser. - Remove the alpha slider from text color controls. Text opacity is already covered by the layer opacity slider, and a standalone alpha slider for a single channel was confusing. The color picker now always sends alpha=255. Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> * fix: preserve existing alpha when changing text color Signed-off-by: Devin AI <devin@devin.ai> Signed-off-by: StreamKit Devin <devin@streamkit.dev> Co-Authored-By: Claudio Costa <cstcld91@gmail.com> --------- Signed-off-by: StreamKit Devin <devin@streamkit.dev> Signed-off-by: Devin AI <devin@devin.ai> Co-authored-by: StreamKit Devin <devin@streamkit.dev> Co-authored-by: Claudio Costa <cstcld91@gmail.com>
1 parent f2ba317 commit b8d9051

File tree

7 files changed

+501
-179
lines changed

7 files changed

+501
-179
lines changed

crates/nodes/src/video/compositor/config.rs

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,14 @@ pub struct TextOverlayConfig {
9494
/// Takes precedence over `font_path` when both are provided.
9595
#[serde(default)]
9696
pub font_data_base64: Option<String>,
97+
/// Named font from a curated set of system fonts.
98+
/// Takes precedence over `font_path` but not `font_data_base64`.
99+
/// Available names: "dejavu-sans", "dejavu-serif", "dejavu-sans-mono",
100+
/// "dejavu-sans-bold", "dejavu-serif-bold", "dejavu-sans-mono-bold",
101+
/// "liberation-sans", "liberation-serif", "liberation-mono",
102+
/// "freesans", "freeserif", "freemono".
103+
#[serde(default)]
104+
pub font_name: Option<String>,
97105
}
98106

99107
pub(crate) const fn default_opacity() -> f32 {

crates/nodes/src/video/compositor/mod.rs

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1106,10 +1106,15 @@ mod tests {
11061106
font_size: 24,
11071107
font_path: None,
11081108
font_data_base64: None,
1109+
font_name: None,
11091110
};
11101111
let overlay = rasterize_text_overlay(&cfg);
1111-
assert_eq!(overlay.width, 64);
1112-
assert_eq!(overlay.height, 32);
1112+
// Width and height should be at least the original rect dimensions.
1113+
assert!(overlay.width >= 64);
1114+
assert!(overlay.height >= 32);
1115+
// The rect in the returned overlay should match the bitmap dimensions.
1116+
assert_eq!(overlay.rect.width, overlay.width);
1117+
assert_eq!(overlay.rect.height, overlay.height);
11131118
// Should have some non-zero pixels (text was drawn).
11141119
assert!(overlay.rgba_data.iter().any(|&b| b > 0));
11151120
}

crates/nodes/src/video/compositor/overlay.rs

Lines changed: 67 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -127,16 +127,47 @@ fn prescale_rgba(src: &[u8], sw: u32, sh: u32, dw: u32, dh: u32) -> Vec<u8> {
127127
/// Path to the system DejaVu Sans font (commonly available on Linux).
128128
const DEJAVU_SANS_PATH: &str = "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf";
129129

130+
/// Map of named fonts to their filesystem paths.
131+
/// All fonts listed here are open-source and royalty-free, commonly installed
132+
/// on Debian/Ubuntu systems via the `fonts-dejavu-core`, `fonts-liberation`,
133+
/// and `fonts-freefont-ttf` packages.
134+
const KNOWN_FONTS: &[(&str, &str)] = &[
135+
("dejavu-sans", "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf"),
136+
("dejavu-serif", "/usr/share/fonts/truetype/dejavu/DejaVuSerif.ttf"),
137+
("dejavu-sans-mono", "/usr/share/fonts/truetype/dejavu/DejaVuSansMono.ttf"),
138+
("dejavu-sans-bold", "/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf"),
139+
("dejavu-serif-bold", "/usr/share/fonts/truetype/dejavu/DejaVuSerif-Bold.ttf"),
140+
("dejavu-sans-mono-bold", "/usr/share/fonts/truetype/dejavu/DejaVuSansMono-Bold.ttf"),
141+
("liberation-sans", "/usr/share/fonts/truetype/liberation/LiberationSans-Regular.ttf"),
142+
("liberation-serif", "/usr/share/fonts/truetype/liberation/LiberationSerif-Regular.ttf"),
143+
("liberation-mono", "/usr/share/fonts/truetype/liberation/LiberationMono-Regular.ttf"),
144+
("freesans", "/usr/share/fonts/truetype/freefont/FreeSans.ttf"),
145+
("freeserif", "/usr/share/fonts/truetype/freefont/FreeSerif.ttf"),
146+
("freemono", "/usr/share/fonts/truetype/freefont/FreeMono.ttf"),
147+
];
148+
130149
/// Load font data, trying (in order):
131150
/// 1. `font_data_base64` (inline base64-encoded TTF/OTF)
132-
/// 2. `font_path` (filesystem path)
133-
/// 3. Bundled system default (`DejaVuSans.ttf`)
151+
/// 2. `font_name` (named font from [`KNOWN_FONTS`] map)
152+
/// 3. `font_path` (filesystem path)
153+
/// 4. Bundled system default (`DejaVuSans.ttf`)
134154
fn load_font(config: &TextOverlayConfig) -> Result<fontdue::Font, String> {
135155
let font_bytes: Vec<u8> = if let Some(ref b64) = config.font_data_base64 {
136156
use base64::Engine;
137157
base64::engine::general_purpose::STANDARD
138158
.decode(b64)
139159
.map_err(|e| format!("Invalid base64 in font_data_base64: {e}"))?
160+
} else if let Some(ref name) = config.font_name {
161+
let path =
162+
KNOWN_FONTS.iter().find(|(n, _)| *n == name.as_str()).map(|(_, p)| *p).ok_or_else(
163+
|| format!("Unknown font name '{name}'. Available: {}", known_font_names()),
164+
)?;
165+
std::fs::read(path).map_err(|e| {
166+
tracing::warn!(
167+
"Named font '{name}' not found at '{path}': {e}. Is the font package installed?"
168+
);
169+
format!("Failed to read named font '{name}' at '{path}': {e}")
170+
})?
140171
} else if let Some(ref path) = config.font_path {
141172
std::fs::read(path).map_err(|e| format!("Failed to read font file '{path}': {e}"))?
142173
} else {
@@ -148,14 +179,19 @@ fn load_font(config: &TextOverlayConfig) -> Result<fontdue::Font, String> {
148179
.map_err(|e| format!("Failed to parse font: {e}"))
149180
}
150181

182+
/// Comma-separated list of available font names for error messages.
183+
fn known_font_names() -> String {
184+
KNOWN_FONTS.iter().map(|(n, _)| *n).collect::<Vec<_>>().join(", ")
185+
}
186+
151187
/// Rasterize a text overlay into an RGBA8 bitmap using `fontdue` for real
152188
/// font glyph rendering. Falls back to solid-rectangle placeholders when
153189
/// font loading fails so the node keeps running.
190+
///
191+
/// The bitmap dimensions are expanded to fit the measured text size so that
192+
/// neither the width nor the height clips the rendered string.
154193
#[allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss)]
155194
pub fn rasterize_text_overlay(config: &TextOverlayConfig) -> DecodedOverlay {
156-
let w = config.transform.rect.width.max(1);
157-
let h = config.transform.rect.height.max(1);
158-
159195
// Attempt to load the font; fall back to rectangle placeholders on error.
160196
let font = match load_font(config) {
161197
Ok(f) => Some(f),
@@ -165,6 +201,24 @@ pub fn rasterize_text_overlay(config: &TextOverlayConfig) -> DecodedOverlay {
165201
},
166202
};
167203

204+
let font_size = config.font_size.max(1) as f32;
205+
206+
// Measure actual text dimensions so the bitmap is large enough to hold
207+
// the full rendered string without clipping.
208+
let (measured_w, measured_h) = font.as_ref().map_or_else(
209+
|| {
210+
// Fallback estimate for placeholder rectangles.
211+
let glyph_w = config.font_size.max(1) * 3 / 5;
212+
let est_w = glyph_w * config.text.chars().count() as u32;
213+
let est_h = (font_size * 1.4).ceil() as u32;
214+
(est_w, est_h)
215+
},
216+
|f| crate::video::measure_text(f, font_size, &config.text),
217+
);
218+
219+
let w = config.transform.rect.width.max(measured_w).max(1);
220+
let h = config.transform.rect.height.max(measured_h).max(1);
221+
168222
let total_bytes = (w as usize) * (h as usize) * 4;
169223
let mut rgba_data = vec![0u8; total_bytes];
170224

@@ -209,7 +263,14 @@ pub fn rasterize_text_overlay(config: &TextOverlayConfig) -> DecodedOverlay {
209263
rgba_data,
210264
width: w,
211265
height: h,
212-
rect: config.transform.rect.clone(),
266+
rect: {
267+
// Use the expanded dimensions so the blit renders the full bitmap
268+
// without clipping text that exceeds the original rect.
269+
let mut r = config.transform.rect.clone();
270+
r.width = w;
271+
r.height = h;
272+
r
273+
},
213274
opacity: config.transform.opacity,
214275
rotation_degrees: config.transform.rotation_degrees,
215276
z_index: config.transform.z_index,

crates/nodes/src/video/mod.rs

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,52 @@ pub mod vp9;
4242

4343
// ── Shared font-rendering helpers ────────────────────────────────────────────
4444

45+
/// Measure the pixel dimensions a single-line text string would occupy when
46+
/// rendered at `font_size`. Returns `(width, height)`.
47+
///
48+
/// The width is the sum of advance widths. The height uses the same baseline
49+
/// logic as [`blit_text_rgba`] and adds enough room for descenders.
50+
#[allow(
51+
clippy::cast_possible_truncation,
52+
clippy::cast_possible_wrap,
53+
clippy::cast_sign_loss,
54+
clippy::cast_precision_loss
55+
)]
56+
pub fn measure_text(font: &fontdue::Font, font_size: f32, text: &str) -> (u32, u32) {
57+
if text.is_empty() {
58+
return (0, 0);
59+
}
60+
61+
let (ref_metrics, _) = font.rasterize('A', font_size);
62+
let baseline_y = ref_metrics.height as f32;
63+
64+
let mut total_width: f32 = 0.0;
65+
let mut max_top: i32 = 0; // highest pixel above origin_y (always >= 0)
66+
let mut max_bottom: i32 = 0; // lowest pixel below origin_y
67+
68+
for ch in text.chars() {
69+
let (metrics, _) = font.rasterize(ch, font_size);
70+
71+
let gy = (baseline_y - metrics.ymin as f32) as i32 - metrics.height as i32;
72+
let glyph_bottom = gy + metrics.height as i32;
73+
74+
if gy < max_top {
75+
max_top = gy;
76+
}
77+
if glyph_bottom > max_bottom {
78+
max_bottom = glyph_bottom;
79+
}
80+
81+
total_width += metrics.advance_width;
82+
}
83+
84+
let w = total_width.ceil() as u32;
85+
let h =
86+
if max_bottom > max_top { (max_bottom - max_top) as u32 } else { font_size.ceil() as u32 };
87+
88+
(w, h)
89+
}
90+
4591
/// Alpha-blend a single text string into a packed RGBA8 buffer.
4692
///
4793
/// `origin_x` / `origin_y` are the top-left pixel coordinates where the first

ui/src/components/CompositorCanvas.tsx

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,35 @@ function layerHue(index: number): number {
3434
return (index * 137.508) % 360;
3535
}
3636

37+
// ── Font mapping ────────────────────────────────────────────────────────────
38+
39+
/** Map backend font names to CSS font-family values for canvas preview.
40+
* These use web-safe fallback stacks so the preview is visually
41+
* representative even when the exact system font is not installed. */
42+
const FONT_FAMILY_MAP: Record<string, string> = {
43+
'dejavu-sans': '"DejaVu Sans", "Verdana", sans-serif',
44+
'dejavu-serif': '"DejaVu Serif", "Georgia", serif',
45+
'dejavu-sans-mono': '"DejaVu Sans Mono", "Courier New", monospace',
46+
'dejavu-sans-bold': '"DejaVu Sans", "Verdana", sans-serif',
47+
'dejavu-serif-bold': '"DejaVu Serif", "Georgia", serif',
48+
'dejavu-sans-mono-bold': '"DejaVu Sans Mono", "Courier New", monospace',
49+
'liberation-sans': '"Liberation Sans", "Arial", sans-serif',
50+
'liberation-serif': '"Liberation Serif", "Times New Roman", serif',
51+
'liberation-mono': '"Liberation Mono", "Courier New", monospace',
52+
freesans: '"FreeSans", "Arial", sans-serif',
53+
freeserif: '"FreeSerif", "Times New Roman", serif',
54+
freemono: '"FreeMono", "Courier New", monospace',
55+
};
56+
57+
/** Return true when the backend font name maps to a bold variant. */
58+
function isBoldFont(fontName: string): boolean {
59+
return fontName.endsWith('-bold');
60+
}
61+
62+
function cssFontFamily(fontName: string): string {
63+
return FONT_FAMILY_MAP[fontName] ?? 'sans-serif';
64+
}
65+
3766
// ── Styled components ───────────────────────────────────────────────────────
3867

3968
const CanvasOuter = styled.div`
@@ -356,15 +385,19 @@ const TextOverlayLayer: React.FC<{
356385
onChange={(e) => setEditText(e.target.value)}
357386
onBlur={commitEdit}
358387
onKeyDown={handleKeyDown}
359-
style={{ fontSize: Math.max(10, overlay.fontSize * scale * 0.6) }}
388+
style={{
389+
fontSize: Math.max(10, overlay.fontSize * scale * 0.6),
390+
fontFamily: cssFontFamily(overlay.fontName),
391+
}}
360392
/>
361393
) : (
362394
<TextContent>
363395
<span
364396
style={{
365397
fontSize: Math.max(8, overlay.fontSize * scale),
366398
color: textColor,
367-
fontWeight: 600,
399+
fontFamily: cssFontFamily(overlay.fontName),
400+
fontWeight: isBoldFont(overlay.fontName) ? 700 : 600,
368401
textShadow: '0 1px 3px rgba(0,0,0,0.7)',
369402
lineHeight: 1.2,
370403
textAlign: 'center',

0 commit comments

Comments
 (0)