Skip to content

feat: Merge v4 into master#70

Open
peterkelly wants to merge 131 commits intomasterfrom
v4
Open

feat: Merge v4 into master#70
peterkelly wants to merge 131 commits intomasterfrom
v4

Conversation

@peterkelly
Copy link
Collaborator

Note: Currently version no. is at 3.9.1; will bump to 4.0.0 once the API has been fully stabilsed (module/library rename done).

This merge will make it easier for others to start using the new version in the meantime, by just referencing the 3.9.x versions.

Fail evaluation earlier when source still contains typed holes by
surfacing a type error during engine type checking instead of letting
execution reach an unsupported-expression runtime failure.

Emit explicit diagnostics for typed holes in the LSP path and wire
hole diagnostics to hole-fill code actions so users are guided toward
repair at the error site.

Add regression coverage for engine hole failure mode, hole diagnostics,
and hole quick-fix availability from real typed-hole diagnostics.
Improve fallback span selection for unspanned type errors in rex-lsp so
diagnostics no longer default to whole-document ranges.

When declaration-oriented type inference fails without a concrete span,
anchor diagnostics to a primary program span instead of Span::default().
Here, primary program span means: use the first declaration span when
declarations exist, otherwise use the top-level expression span.

Add regression coverage to ensure a unification error is not reported as
a full-file range.

Update the LLM docs typed-hole example to use consistent Result type
parameter order and float widths, avoiding misleading unrelated type
errors in the interactive chapter snippet.
Move ADT registration behind a shared trait and engine helpers so manual
Rust types and data-driven schemas can register ADTs without relying on
derive-only macro output.

Introduce RexAdt and type-driven AdtDecl construction APIs, refactor
derive(Rex) to use the shared path, and add integration tests for manual
registration, error paths, and derive parity.
Add the Default type class to the built-in prelude so scripts can use
default without redefining the class.

Use contextual type hints for record-update inference so annotated
contexts can resolve default without forcing is in common let and
annotation flows.

Expand LSP diagnostics and quick fixes for ambiguous default record
updates, including per-ADT is fixes and let-binding annotation fixes,
and add an end-to-end test that validates diagnostics, spans, quick-fix
application, and final evaluation.

Refresh the Defaulting docs to clarify typeclass-vs-numeric meanings,
document ambiguity patterns and fixes, and align wording with LSP
code-action terminology.

Update interactive runtime behavior so Run remains available and, when
type errors exist, it reports type-checking errors instead of trying to
evaluate.
Add host-state-backed defaults for derived ADTs so embedded Rex code can
use default values that come from runtime context. This enables real
embedding workflows where values like account_id and project_id must
come from engine state instead of hardcoded literals.

Introduce RexDefault as the integration point and wire it into runtime
Default resolution. The engine now supports registering a Default
instance for a Rust ADT via inject_rex_default_instance, and the derive
macro now provides inject_rex_with_default so ADT injection and Default
instance registration happen together in one explicit call.

Extend embedding coverage with an Entity example that implements
RexDefault against HostState. Tests now verify both plain default and
record-update usage over default values, confirming the defaults are
host-derived, type-correct, and composable with normal Rex update
expressions.

The separate inject_rex_with_default method is required by Rust trait
system limits: inject_rex cannot conditionally add behavior based on
whether RexDefault is implemented without specialization (or making
RexDefault mandatory for all derived ADTs). This keeps inject_rex
backward-compatible while providing an explicit opt-in path.
Diagnostics printed named type variables as bare identifiers (for
example, a), which looked like let-binding names in docs examples.

Render named type variables in ML style (for example, 'a) so messages
clearly refer to polymorphic type variables rather than program
variables.

Update the LSP assertion strings for the ambiguous default record update
example to match the new rendering.
Support host-provided constructors for derived Rex types where Rex
callers only supply the required fields, while the constructor fills the
remaining fields from host context.

This is aimed at embedding scenarios where records include host-owned
values such as account or project identifiers. Rex scripts can now call
a constructor with only workflow-provided inputs and still produce a
fully populated host type.

This also improves backward compatibility for embedded workflows: host
types can gain additional optional fields without forcing existing Rex
constructor call sites to change.

The change makes this pattern reusable across derived host types, so
embedders can register these constructors directly instead of
hand-writing per-type constructor glue.
This change set makes host-facing collection interop workable without
forcing users to write boilerplate conversions at every call site. The
core reason is practical embedding ergonomics: host APIs naturally use
contiguous vectors for efficiency, while Rex user code relies on list
patterns and list-oriented style. We now support that boundary in a
narrow, intentional way that preserves performance control.

On the type system path, function argument position now accepts list
inputs for host functions expecting arrays by inserting list-to-array
coercion during typing. This avoids combinatorial overload expansion,
keeps existing manual host impls intact, and solves the high-frequency
interop case where users pass list literals to host exports. The inverse
conversion remains explicit via to_list so array-to-list allocation is
never hidden in arbitrary expression contexts.

Prelude and runtime surface were updated to make the conversion model
clear and stable: to_array and to_list are first-class helpers, with
matching docs metadata and generated prelude reference entries. LSP now
includes an array/list mismatch quick fix that suggests wrapping an
expression with to_list. This enables one-pass semantic repair in editor
and automated tooling flows.

The constructor/export plumbing was simplified by removing the
EngineHandler and RexConstructArg split that had been introduced for
constructor defaults. rex_new/rex_default now work off host state, and
constructor injection uses the regular Handler path. This removes a
special-case API tier, cuts per-type boilerplate, and keeps host-side
constructor authoring in conventional Rust style while still supporting
required constructor parameters plus defaulted fields.

Tests and tutorial/embedding docs were expanded to lock behavior and
explain rationale. New tests cover list-literal acceptance for Vec
arguments, pattern matching on host-returned arrays via to_list, and the
Entity2 constructor-default flow. Section ordering and collections
documentation were also adjusted to reflect the new guidance and quick
fix workflow.
Consolidate embedding around a single module-scoped model so host
registration no longer depends on split global versus module code
paths. This reduces conceptual overhead for embedders and avoids
behavior drift between APIs that should represent the same operation.

Represent the prelude as a default-imported module and make that
behavior configurable through engine options. This preserves ergonomic
defaults while giving embedders explicit control for sandboxing,
reproducibility, and custom runtime environments.

Align docs and regression tests with this model so module ADTs and
module-scoped host exports are the primary path. This keeps the
embedding guidance consistent with runtime behavior and locks in the
name-resolution rules for default imports and root-scope exports.
Replace stringly-typed module, type, class, and value reference paths
with structured resolution and canonical symbol mapping across rex-ast,
rex-parser, rex-ts, rex-engine, rex-lsp, and rex CLI JSON entry points.
The previous approach encoded qualified names as flattened strings in
too many core paths, which made namespace semantics implicit and
brittle. The new model makes qualified access explicit by carrying
structured refs through parse, rewrite, validation, inference, and
runtime preparation.

Unify qualified name behavior across expression and type positions so
module alias lookup is applied consistently to expression variables,
constructor references, pattern constructors, type annotations, function
signatures, where constraints, superclass clauses, and instance headers.
In particular, instance class names now support qualified alias form in
the same resolution pipeline as other class references. This removes
position-dependent behavior that previously produced inconsistent
outcomes for equivalent references.

Move import-use validation earlier in module processing and make it
authoritative for qualified alias member checks in type and class
contexts. Missing exports for alias-qualified members are now reported
as module errors before type inference or runtime evaluation. This is a
deliberate shift to fail fast at the module boundary, improve diagnostic
locality, and prevent delayed failures that obscured the real source of
invalid references.

Strengthen module loading semantics around declaration-driven
initialization and cycle support by loading interfaces through SCC
processing rather than import-triggered execution behavior. This
preserves deterministic behavior under cyclic imports, supports
mutually-referential module interfaces, and maintains the design goal
that imports do not execute arbitrary top-level expressions. The module
system now resolves and rewrites against stable interface data before
downstream phases consume declarations.

Fix ordering-sensitive rewrite edge cases around lexical binders and
imported aliases so annotation resolution remains correct when binder
names and aliases overlap. Binder introduction no longer incorrectly
suppresses alias resolution inside its own type context, and rewrite
order now matches declaration semantics instead of incidental traversal
order. This prevents subtle HM-context regressions that surfaced in
adversarial shadowing cases.

Adopt BuiltinTypeId-backed builtin type construction across
runtime-facing paths, prelude wiring, host-facing examples, and related
tests in place of ad hoc string constructor names for builtins. This
reduces collision risk with user-defined names, clarifies builtin
identity in core APIs, and aligns builtin representation with the
canonical symbol direction introduced in this change. The resulting API
surface is more explicit and safer for embedding scenarios.

Refactor module and engine internals to carry structured export maps and
canonical symbols for values, types, and classes, then use those maps in
import binding, rewrite, and validation passes. Update LSP program
preparation and diagnostics to mirror engine semantics so editor
feedback and runtime behavior agree for qualified references, missing
export errors, and rewritten internal names. This avoids a split-brain
model where tooling and execution diverge.

Expand regression coverage with adversarial tests across parser, type
system, engine, module loader, LSP diagnostics, and integration suites.
New and updated tests lock behavior for qualified class names in
instance headers, qualified type and class validation in signatures and
constraints, constructor rewrite paths in patterns, alias shadowing
order, cycle handling, and canonicalized builtin type usage. This
breadth is intentional because the change touches semantic
infrastructure shared by nearly every phase of the pipeline.

Refresh documentation to match shipped semantics and remove ambiguity
for embedders and language users. Update ARCHITECTURE, EMBEDDING,
LANGUAGE, SPEC, and tutorial sections on instances and constraints to
document declaration-only module files, import clause versus alias
namespace behavior, qualified type and class access, qualified instance
headers, and early module-error validation for missing alias exports.
Documentation now describes the same behavior enforced by tests and
implementation, reducing drift between reference text and actual
semantics.
@peterkelly peterkelly requested a review from loongy March 24, 2026 06:46
…n permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants