fix(smrt-core): prefer huggingface transformers for embeddings#1109
fix(smrt-core): prefer huggingface transformers for embeddings#1109willgriffin merged 2 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates @happyvertical/smrt-core local embedding initialization to prefer @huggingface/transformers (when installed) and fall back to @xenova/transformers, while improving missing-package errors and adding regression tests for module resolution behavior.
Changes:
- Add runtime resolution logic to prefer
@huggingface/transformersfor local embeddings with fallback to@xenova/transformers. - Improve the “missing optional dependency” error message and preserve non-module runtime failures.
- Add Vitest coverage for the package resolution order and error behavior.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Adds @huggingface/transformers to core dev deps (lockfile needs to be consistent with package.json). |
| packages/core/src/embeddings/provider.ts | Implements optional transformers package resolution and uses it for local pipeline initialization and availability checks. |
| packages/core/src/tests/embedding-provider-transformers-resolution.test.ts | Adds regression tests for preference order, fallback, and error propagation. |
| packages/core/package.json | Declares @huggingface/transformers as an optional peer dependency and adds it to devDependencies. |
| .changeset/fair-squids-tickle.md | Patch changeset documenting the behavior change and Node 24/sharp motivation. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| '@huggingface/transformers': | ||
| specifier: ^3.8.1 | ||
| version: 3.8.1 | ||
| '@types/node': | ||
| specifier: 24.10.9 | ||
| version: 24.10.9 |
There was a problem hiding this comment.
In the lockfile, packages/core still pins @types/node to 24.10.9, but packages/core/package.json specifies @types/node: "25.0.9". Please regenerate/update pnpm-lock.yaml so the recorded specifier/version matches the package.json dependency set (usually by running pnpm install from the repo root).
| if (lastError) { | ||
| throw formatTransformersResolutionError(LOCAL_TRANSFORMERS_PACKAGES); | ||
| } | ||
|
|
||
| throw formatTransformersResolutionError(LOCAL_TRANSFORMERS_PACKAGES); |
There was a problem hiding this comment.
The final if (lastError) { throw ... } followed by an unconditional throw ... is redundant here, since both branches throw the same error. Consider removing the conditional (and potentially the unused lastError tracking) to simplify the control flow.
Summary
@huggingface/transformersfor local embeddings when available, with fallback to@xenova/transformersVerification