refactor(sdk): migrated by lloyal-research · Pull Request #25 · lloyal-ai/lloyal.node

lloyal-research · 2026-03-05T09:09:57Z

No description provided.

Copilot

Pull request overview

This PR migrates @lloyal-labs/lloyal.node to be a thin native-backend package that delegates inference primitives and the agents framework to the new @lloyal-labs/sdk and @lloyal-labs/lloyal-agents packages, while updating tests/docs accordingly.

Changes:

Re-export SDK primitives/types from @lloyal-labs/sdk and agents APIs from @lloyal-labs/lloyal-agents, keeping only native loading/context creation in this package.
Remove the in-repo implementations for Branch/BranchStore/Session/Rerank and the in-repo agents runtime (moved to external packages).
Update integration tests and README to reflect the new package split; remove the deep-research example and related tests/config.

Reviewed changes

Copilot reviewed 41 out of 42 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/integration.ts	Updates rerank tests to use `createContext()` + new `Rerank.create(ctx, opts)` shape.
test/examples.ts	Removes deep-research example test wiring.
test/agents.ts	Removes structured-concurrency agent tests and their runner script.
src/types.ts	Shrinks types to “native-only” surface and imports shared types from `@lloyal-labs/sdk`.
src/index.ts	Re-exports SDK + agents APIs; keeps `loadBinary()` / `createContext()` as the native entrypoints.
src/agents/types.ts	Deleted (agents types moved out to `@lloyal-labs/lloyal-agents`).
src/agents/toolkit.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/shared-root.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/run-agents.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/init.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/index.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/generate.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/diverge.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/deltas.ts	Deleted (moved out to `@lloyal-labs/sdk` / `@lloyal-labs/lloyal-agents`).
src/agents/context.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/agent-pool.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/agents/Tool.ts	Deleted (moved out to `@lloyal-labs/lloyal-agents`).
src/Session.ts	Deleted (Session moved to `@lloyal-labs/sdk`).
src/Rerank.ts	Deleted (Rerank moved to `@lloyal-labs/sdk`).
src/BranchStore.ts	Deleted (BranchStore moved to `@lloyal-labs/sdk`).
src/Branch.ts	Deleted (Branch moved to `@lloyal-labs/sdk`).
package.json	Adds dependencies on `@lloyal-labs/sdk` and `@lloyal-labs/lloyal-agents`; removes direct effection dependency.
package-lock.json	Updates dependency graph, but currently pins SDK packages via local `../lloyal-sdk/...` links.
examples/deep-research/tui.ts	Deleted (deep-research example removed).
examples/deep-research/tools/types.ts	Deleted (deep-research example removed).
examples/deep-research/tools/search.ts	Deleted (deep-research example removed).
examples/deep-research/tools/report.ts	Deleted (deep-research example removed).
examples/deep-research/tools/read-file.ts	Deleted (deep-research example removed).
examples/deep-research/tools/index.ts	Deleted (deep-research example removed).
examples/deep-research/tools/grep.ts	Deleted (deep-research example removed).
examples/deep-research/tasks/verify.md	Deleted (deep-research example removed).
examples/deep-research/tasks/research.md	Deleted (deep-research example removed).
examples/deep-research/tasks/report.md	Deleted (deep-research example removed).
examples/deep-research/tasks/plan.md	Deleted (deep-research example removed).
examples/deep-research/tasks/eval.md	Deleted (deep-research example removed).
examples/deep-research/resources/types.ts	Deleted (deep-research example removed).
examples/deep-research/resources/files.ts	Deleted (deep-research example removed).
examples/deep-research/reranker.ts	Deleted (deep-research example removed).
examples/deep-research/main.ts	Deleted (deep-research example removed).
examples/deep-research/harness.ts	Deleted (deep-research example removed).
examples/deep-research/agreement.ts	Deleted (deep-research example removed).
README.md	Rewrites docs to describe the package as a native backend + re-export hub for SDK/agents.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-05T09:17:01Z

src/types.ts

 /**
- * liblloyal-node TypeScript Definitions
+ * liblloyal-node — native-only type definitions
 *
- * N-API bindings for liblloyal - Node.js native addon for llama.cpp inference
- *
- * @categoryDescription Core
- * Entry points, context lifecycle, and the main inference interface.
- *
- * @categoryDescription Sampling
- * Sampler chain configuration — temperature, penalties, nucleus sampling, and advanced filters.
- *
- * @categoryDescription Chat
- * Chat template formatting, output parsing, tool calls, and reasoning extraction.
- *
- * @categoryDescription Branching
- * Parallel and tree-structured generation with batched GPU dispatch.
+ * Types specific to the Node.js native addon (binary loading, GPU variant
+ * selection). All inference primitives and shared types are in
+ * {@link @lloyal-labs/sdk | @lloyal-labs/sdk}.
 */


This file header says lloyal.node is "native-only" and that all inference primitives/types are in @lloyal-labs/sdk. However the public API still exposes GPU variant selection, and on some platforms the "default" binary is GPU-backed (e.g. Metal on macOS arm64). Consider clarifying in this header (or in the GpuVariant docs) what "default" means per-platform to avoid implying CPU-only behavior.

Copilot · 2026-03-05T09:17:01Z

README.md

+const { prompt, grammar } = await ctx.formatChat(messages, {
+  addGenerationPrompt: true,
+  tools: [{ type: "function", function: { name: "search", parameters: schema } }],
+});
+const { content, toolCalls } = await ctx.parseChatOutput(output);


In this snippet, parseChatOutput is called with only the output string. In the rest of the repo (e.g. examples/chat/chat.ts and test/integration.ts) it is called as parseChatOutput(output, fmt.format, opts) using the format returned from formatChat(). Please update this example to pass the required format (and any needed options), otherwise readers will copy an API call that doesn't match the actual SessionContext interface.

Suggested change

const { prompt, grammar } = await ctx.formatChat(messages, {

addGenerationPrompt: true,

tools: [{ type: "function", function: { name: "search", parameters: schema } }],

});

const { content, toolCalls } = await ctx.parseChatOutput(output);

const { prompt, grammar, format } = await ctx.formatChat(messages, {

addGenerationPrompt: true,

tools: [{ type: "function", function: { name: "search", parameters: schema } }],

});

const { content, toolCalls } = await ctx.parseChatOutput(output, format);

Copilot · 2026-03-05T09:17:02Z

README.md

+// Branch primitives — what the SDK's Branch class wraps
+const handle = ctx._branchCreate(0, samplerParams);
+await ctx._branchPrefill(handle, tokens);
+const { token, text, isStop } = ctx._branchSample(handle);


ctx._branchSample(handle) returns a token id (number) in the native binding (see src/SessionContext.cpp), not an object containing { token, text, isStop }. This example should either call _branchSample and then derive text/isStop via tokenToText() + isStopToken(), or use the higher-level Branch.produceSync() API.

Suggested change

const { token, text, isStop } = ctx._branchSample(handle);

const token = ctx._branchSample(handle);

const text = ctx.tokenToText(token);

const isStop = ctx.isStopToken(token);

Copilot · 2026-03-05T09:17:02Z

README.md

+await ctx._storeCommit([[handle1, tok1], [handle2, tok2]]);  // N branches, 1 GPU call
+await ctx._storePrefill([[handle, tokens]]);


The low-level store APIs are documented here as taking arrays of [handle, token] tuples, but the native addon methods _storeCommit / _storePrefill actually require separate handles[] + tokens[] / tokenArrays[][] parameters (see src/SessionContext.cpp error messages). Please update the example call signatures to match the real N-API interface to prevent copy/paste breakage.

Suggested change

await ctx._storeCommit([[handle1, tok1], [handle2, tok2]]); // N branches, 1 GPU call

await ctx._storePrefill([[handle, tokens]]);

await ctx._storeCommit([handle1, handle2], [tok1, tok2]); // N branches, 1 GPU call

await ctx._storePrefill([handle], [tokens]);

refactor(sdk): migrated

b32b41a

Copilot AI review requested due to automatic review settings March 5, 2026 09:09

Copilot started reviewing on behalf of lloyal-research March 5, 2026 09:10 View session

Copilot AI reviewed Mar 5, 2026

View reviewed changes

lloyal-research added 2 commits March 6, 2026 15:26

feat(docs): update README

6a47b1f

feat(ci): update deps

db7dd60

lloyal-research merged commit cb970e1 into main Mar 6, 2026
7 checks passed

lloyal-research deleted the refactor/sdk-migration branch March 17, 2026 05:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(sdk): migrated#25

refactor(sdk): migrated#25
lloyal-research merged 3 commits intomainfrom
refactor/sdk-migration

lloyal-research commented Mar 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-const { token, text, isStop } = ctx._branchSample(handle);
+const token = ctx._branchSample(handle);
+const text = ctx.tokenToText(token);
+const isStop = ctx.isStopToken(token);

		await ctx._storeCommit([[handle1, tok1], [handle2, tok2]]); // N branches, 1 GPU call
		await ctx._storePrefill([[handle, tokens]]);

Conversation

lloyal-research commented Mar 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants