Feat/api expansion

[lloyal-research]

No description provided.

[greptile-apps]

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

[lloyal-research]

here
renewed, please complete the review

[greptile-apps]

Greptile Overview

Greptile Summary

This PR adds comprehensive embedding extraction functionality to lloyal.node, expanding the API to support text embedding models alongside the existing text generation capabilities. The implementation includes:

Major Changes:

• Added embedding extraction APIs (encode, getEmbeddings, getEmbeddingDimension, hasPooling) with L2 normalization support
• Integrated upstream liblloyal updates including new embedding.hpp, kv.hpp, and tokenizer.hpp headers
• Added comprehensive embedding test suite with 8 tests covering pooling, normalization, and similarity
• Updated CI to run embedding tests across all platforms
• Replaced complex CLI example with simpler chat and embed examples
• Removed vendored liblloyal test suite (only headers needed for building)

Architecture:

• Follows the same async worker pattern as existing APIs for encode operations
• Uses header-only liblloyal utilities for embedding extraction and normalization
• Properly handles pooling types (NONE, MEAN, CLS, LAST) via context parameters
• Maintains zero-copy access patterns where appropriate

Testing:

• Gracefully skips embedding tests when model unavailable (CI-friendly)
• Tests L2 normalization, cosine similarity, and semantic relationships
• Includes practical examples demonstrating embedding workflows

Confidence Score: 4/5

• Safe to merge with high confidence - well-tested embedding functionality with proper async handling
• Score reflects solid implementation with comprehensive tests and examples. Minor concerns about buffer lifetime in getEmbeddings (copies data, so safe) and the large vendor file deletions (test files removed as they're build-only dependencies). The async worker pattern is correctly applied, error handling is thorough, and the CI integration ensures cross-platform compatibility.
• No files require special attention - the implementation follows established patterns and includes appropriate safety checks

Important Files Changed

File Analysis

Filename	Score	Overview
src/SessionContext.cpp	4/5	Added embedding extraction APIs (encode, getEmbeddings, getEmbeddingDimension, hasPooling) with proper async workers and L2 normalization support
lib/index.d.ts	5/5	Comprehensive TypeScript definitions for embedding APIs with detailed JSDoc documentation including PoolingType enum and embedding-related methods
test/embedding.js	5/5	Well-structured embedding test suite with 8 tests covering context creation, pooling, dimension validation, L2 normalization, and cosine similarity
vendor/liblloyal/include/lloyal/embedding.hpp	5/5	New header-only embedding utilities with L2 normalization, pooling checks, and multiple extraction methods (sequence-specific, per-token)
.github/workflows/tests.yml	5/5	Added embedding tests to CI pipeline with proper timeout and PATH configuration for all platforms

Sequence Diagram

sequenceDiagram
    participant User as User Code
    participant Addon as SessionContext (N-API)
    participant Worker as EncodeWorker
    participant Lloyal as liblloyal::embedding
    participant Llama as llama.cpp

    Note over User,Llama: Context Creation with Embedding Mode
    User->>Addon: createContext({embeddings: true, poolingType: MEAN})
    Addon->>Llama: llama_init_from_model(ctx_params.embeddings=true)
    Llama-->>Addon: llama_context*
    Addon-->>User: SessionContext

    Note over User,Llama: Embedding Extraction Flow
    User->>Addon: tokenize("Hello world")
    Addon->>Worker: TokenizeWorker
    Worker->>Lloyal: lloyal::tokenizer::tokenize()
    Lloyal->>Llama: llama_tokenize()
    Llama-->>Lloyal: tokens[]
    Lloyal-->>Worker: tokens[]
    Worker-->>Addon: Promise resolved
    Addon-->>User: tokens[]

    User->>Addon: kvCacheClear()
    Addon->>Worker: KVCacheClearWorker
    Worker->>Lloyal: lloyal::kv::clear_all()
    Lloyal->>Llama: llama_memory_clear()
    Llama-->>Lloyal: success
    Lloyal-->>Worker: void
    Worker-->>Addon: Promise resolved
    Addon-->>User: void

    User->>Addon: encode(tokens)
    Addon->>Worker: EncodeWorker
    Worker->>Lloyal: lloyal::decoder::encode()
    Note over Lloyal: Marks ALL tokens with logits=true
    Lloyal->>Llama: llama_decode()
    Llama-->>Lloyal: success
    Lloyal-->>Worker: void
    Worker-->>Addon: Promise resolved
    Addon-->>User: void

    User->>Addon: getEmbeddings(normalize=true)
    Addon->>Lloyal: lloyal::embedding::get(ctx, L2)
    Lloyal->>Llama: llama_get_embeddings_seq(ctx, 0)
    Llama-->>Lloyal: float* embeddings
    Lloyal->>Lloyal: apply_l2_normalize()
    Lloyal-->>Addon: vector<float>
    Addon-->>User: Float32Array (L2-normalized)

    Note over User,Llama: Dimension and Capability Checks
    User->>Addon: getEmbeddingDimension()
    Addon->>Lloyal: lloyal::embedding::dimension()
    Lloyal->>Llama: llama_model_n_embd()
    Llama-->>Lloyal: int32_t
    Lloyal-->>Addon: dimension
    Addon-->>User: number

    User->>Addon: hasPooling()
    Addon->>Lloyal: lloyal::embedding::has_pooling()
    Lloyal->>Llama: llama_pooling_type()
    Llama-->>Lloyal: pooling_type
    Lloyal-->>Addon: bool
    Addon-->>User: boolean

Loading

[greptile-apps]

_{27 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}