build: optimize Rust build caching with sccache, shared plugin target dir, and cargo-sweep (#224)

* build: optimize Rust build caching with sccache, shared plugin target dir, and cargo-sweep

- Add .cargo/config.toml with sccache as default rustc-wrapper for
  transparent compilation caching across cargo clean cycles
- Add shared CARGO_TARGET_DIR for all native plugin builds, deduplicating
  common dependency compilation (serde, tokio, etc.) across 11 plugins
- Add 'just sweep' recipe as a lightweight alternative to cargo clean,
  pruning only stale artifacts older than N days (default: 7)
- Update all plugin build, lint, fix, upload, and copy recipes to use
  the shared target/plugins/ directory
- Update plugin.yml artifact paths, official-plugins.json, docs, READMEs,
  and build_official_plugins.sh to reflect new shared target layout

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix(build): comment out sccache in .cargo/config.toml to avoid breaking CI

CI runners don't have sccache installed, so a hard-coded rustc-wrapper
breaks all cargo invocations. Keep the config as a commented-out opt-in
that developers can enable locally after installing sccache.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix(build): set CARGO_TARGET_DIR in sweep recipe for plugin loop

The plugin sweep loop was running cargo sweep without CARGO_TARGET_DIR,
targeting nonexistent per-plugin target/ dirs instead of the shared
target/plugins/ directory.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix(ci): update marketplace-build rust-cache for shared plugin target dir

The marketplace build now uses CARGO_TARGET_DIR=target/plugins, so the
rust-cache workspace config needs to point to the shared target dir
instead of per-plugin workspace entries.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix(build): update gen-docs-reference and marketplace CI for shared plugin target dir

- gen-docs-reference.rs: also search target/plugins/release/ for plugin
  artifacts alongside the legacy per-plugin target dirs
- marketplace-build.yml: restore dynamic workspace collection but point
  each plugin workspace to the shared target dir (../../../target/plugins)
  so rust-cache correctly hashes plugin Cargo.lock files for cache keys

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* refactor(build): simplify sweep to single invocation on shared plugin target dir

Since all plugins share target/plugins/, the per-plugin loop was
redundant — each iteration swept the same directory. Replace with a
single cargo sweep call using CARGO_TARGET_DIR.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix(build): deduplicate plugin artifacts by filename in gen-docs-reference

During transition from per-plugin to shared target dir, both paths may
contain the same plugin .so. Add dedup_by filename after sort+dedup to
prevent duplicate plugin entries in generated docs.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix(build): use HashSet for filename dedup in gen-docs-reference

dedup_by only removes consecutive duplicates, but same-filename paths
from different directories are not adjacent after sorting by full path.
Use a HashSet to properly deduplicate by filename regardless of order.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* style: format gen-docs-reference.rs

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix(build): use is_some_and instead of map_or to satisfy clippy

Replace map_or(false, ...) with is_some_and(...) per
clippy::unnecessary_map_or lint.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* style: format gen-docs-reference.rs

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* ci: install sccache on all CI runners via mozilla-actions/sccache-action

Enable sccache as the default rustc wrapper in .cargo/config.toml
(previously commented out) now that all CI jobs that compile Rust have
mozilla-actions/sccache-action@v0.0.9 installed. This caches compiled
crate artifacts by input hash, making rebuilds after cargo clean
near-instant for cached crates.

Affected workflows: skit.yml (lint, test, test-gpu, build),
plugins.yml (lint-simple, lint-whisper, lint-nllb, lint-sherpa),
e2e.yml, release.yml, marketplace-build.yml.

The format job in plugins.yml is excluded since cargo fmt doesn't
invoke rustc and thus doesn't need the wrapper.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* docs: add sccache as a prerequisite in CONTRIBUTING.md

Document sccache as a required build tool alongside Rust, Bun, and Just.
Include installation command and RUSTC_WRAPPER escape hatch tip.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

* fix: address review findings for build caching PR

- Make sccache env-based opt-in instead of hard requirement in
  .cargo/config.toml (CI sets RUSTC_WRAPPER via sccache-action)
- Document cargo-sweep as optional prerequisite in CONTRIBUTING.md
- Add guard to sweep recipe checking for cargo-sweep installation
- Remove no-op CARGO_TARGET_DIR from cargo fmt invocations
- Add TODO comment for legacy per-plugin target search removal

Signed-off-by: Devin AI <devin@streamkit.dev>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>

---------

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Signed-off-by: Devin AI <devin@streamkit.dev>
Co-authored-by: StreamKit Devin <devin@streamkit.dev>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>

Author: 3 people

.cargo/config.toml

@@ -0,0 +1,10 @@
+# SPDX-FileCopyrightText: © 2025 StreamKit Contributors
+#
+# SPDX-License-Identifier: MPL-2.0
+
+# sccache is recommended for faster rebuilds. CI enables it automatically
+# via mozilla-actions/sccache-action. For local development, install sccache
+# and export RUSTC_WRAPPER=sccache in your shell (see CONTRIBUTING.md).
+# Uncomment below to enable it repo-wide instead of via environment:
+# [build]
+# rustc-wrapper = "sccache"

.github/workflows/e2e.yml

@@ -44,6 +44,8 @@ jobs:
         with:
           toolchain: "1.92.0"
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
 
       - name: Install system dependencies (libvpx for VP9, nasm for AV1 assembly)

.github/workflows/marketplace-build.yml

@@ -85,6 +85,8 @@ jobs:
         with:
           toolchain: "1.92.0"
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - name: Verify official plugins metadata is current
         run: |
           python3 scripts/marketplace/generate_official_plugins.py
@@ -110,7 +112,7 @@ jobs:
           with open(out_path, "a", encoding="utf-8") as handle:
               handle.write("workspaces<<EOF\n")
               for plugin_id in plugin_ids:
-                  handle.write(f"plugins/native/{plugin_id}\n")
+                  handle.write(f"plugins/native/{plugin_id} -> ../../../target/plugins\n")
               handle.write("EOF\n")
           PY
 

.github/workflows/plugins.yml

@@ -73,6 +73,8 @@ jobs:
           toolchain: "1.92.0"
           components: clippy
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
         with:
           workspaces: |
@@ -101,6 +103,8 @@ jobs:
           toolchain: "1.92.0"
           components: clippy
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
         with:
           workspaces: |
@@ -153,6 +157,8 @@ jobs:
           toolchain: "1.92.0"
           components: clippy
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
         with:
           workspaces: |
@@ -198,6 +204,8 @@ jobs:
           toolchain: "1.92.0"
           components: clippy
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
         with:
           workspaces: |

.github/workflows/release.yml

@@ -36,6 +36,8 @@ jobs:
         with:
           toolchain: "1.92.0"
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
 
       - name: Install system dependencies (libvpx for VP9, nasm for AV1 assembly)

.github/workflows/skit.yml

@@ -45,6 +45,8 @@ jobs:
           toolchain: "1.92.0"
           components: rustfmt, clippy
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
 
       - name: Check formatting
@@ -100,6 +102,8 @@ jobs:
         with:
           toolchain: "1.92.0"
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
 
       - name: Run tests (GPU tests run separately on self-hosted runner)
@@ -122,6 +126,8 @@ jobs:
         with:
           toolchain: "1.92.0"
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
 
       - name: Run GPU tests
@@ -173,6 +179,8 @@ jobs:
         with:
           toolchain: "1.92.0"
 
+      - uses: mozilla-actions/sccache-action@v0.0.9
+
       - uses: Swatinem/rust-cache@v2
 
       - name: Build all crates

CONTRIBUTING.md

@@ -47,6 +47,25 @@ curl -fsSL https://bun.sh/install | bash
 cargo install just
 ```
 
+### sccache (build cache — recommended)
+
+[sccache](https://github.com/mozilla/sccache) caches compiled crate artifacts by input hash, making rebuilds significantly faster. CI uses it automatically; for local development:
+
+```bash
+cargo install sccache --locked
+export RUSTC_WRAPPER=sccache   # add to your shell profile
+```
+
+> **Note:** You can also uncomment the `rustc-wrapper` line in `.cargo/config.toml` to enable it repo-wide instead of via environment variable.
+
+### cargo-sweep (build cleanup — optional)
+
+Used by `just sweep` to prune stale build artifacts without a full `cargo clean`:
+
+```bash
+cargo install cargo-sweep --locked
+```
+
 ### Linting tools
 
 Required by `just lint`:

apps/skit/src/bin/gen-docs-reference.rs

@@ -1293,7 +1293,9 @@ fn find_official_native_plugin_artifacts(repo_root: &Path) -> Result<Vec<PathBuf
             continue;
         }
 
+        // TODO: remove legacy per-plugin target search once transition is complete.
         // Prefer release artifacts; fall back to debug if release isn't present.
+        // Check both per-plugin target dirs (legacy) and the shared target/plugins/ dir.
         let release_dir = plugin_dir.join("target/release");
         let debug_dir = plugin_dir.join("target/debug");
 
@@ -1304,6 +1306,15 @@ fn find_official_native_plugin_artifacts(repo_root: &Path) -> Result<Vec<PathBuf
         }
     }
 
+    // Also search the shared plugin target directory (target/plugins/).
+    let shared_release = repo_root.join("target/plugins/release");
+    let shared_debug = repo_root.join("target/plugins/debug");
+    if shared_release.exists() {
+        paths.extend(find_so_files(&shared_release)?);
+    } else if shared_debug.exists() {
+        paths.extend(find_so_files(&shared_debug)?);
+    }
+
     // Filter out debug symbol files (.d), deps folder artifacts, and proc-macro libraries
     // Keep only the main plugin library (e.g., libkokoro.so) not dependency artifacts
     #[allow(clippy::case_sensitive_file_extension_comparisons)]
@@ -1319,6 +1330,11 @@ fn find_official_native_plugin_artifacts(repo_root: &Path) -> Result<Vec<PathBuf
     });
     paths.sort();
     paths.dedup();
+    // Deduplicate by filename to handle the transition period where both
+    // legacy per-plugin target dirs and the shared target/plugins/ dir coexist.
+    // Use a HashSet since same-filename paths are not adjacent after sorting by full path.
+    let mut seen_filenames = std::collections::HashSet::new();
+    paths.retain(|p| p.file_name().is_some_and(|name| seen_filenames.insert(name.to_os_string())));
 
     Ok(paths)
 }

docs/src/content/docs/reference/plugins/plugin-native-helsinki.md

@@ -9,7 +9,7 @@ description: "Neural machine translation using Helsinki-NLP OPUS-MT models. Supp
 
 Neural machine translation using Helsinki-NLP OPUS-MT models. Supports bidirectional EN<->ES translation with Apache 2.0 licensed models. Powered by Candle (pure Rust ML framework).
 
-Source: `plugins/native/helsinki/target/release/libhelsinki.so`
+Source: `target/plugins/release/libhelsinki.so`
 
 ## Categories
 - `ml`

docs/src/content/docs/reference/plugins/plugin-native-kokoro.md

@@ -9,7 +9,7 @@ description: "High-quality text-to-speech synthesis using the Kokoro TTS model.
 
 High-quality text-to-speech synthesis using the Kokoro TTS model. Supports 103 voices across Chinese and English with streaming output. Outputs 24kHz mono audio for real-time playback or further processing.
 
-Source: `plugins/native/kokoro/target/release/libkokoro.so`
+Source: `target/plugins/release/libkokoro.so`
 
 ## Categories
 - `audio`