Skip to content

Commit 319880a

Browse files
KrisSimonKrisSimon
committed
docs(book): add Chapter 11 - Dual-Mode Execution Parity
Documents the interpreter/binary divergence problem, all fixes applied in this session, and the architectural patterns to prevent future drift: - VerbSets shared module (single source of truth for verb classification) - Integer division parity fix (Int/Int → Int in both modes) - DomainEvent co-publishing pattern with full payload schema table - Handler registration template (aro_runtime_register_* pattern) - when-guard serialization: interpreter uses ExpressionEvaluator, binary serializes to JSON and evaluates via evaluateExpressionJSON - mode: both coverage: 81/85 examples, 4 interpreter-only with issues - Verification checklist for future event type additions Also updates STRUCTURE.md table of contents.
1 parent 359f184 commit 319880a

File tree

2 files changed

+299
-0
lines changed

2 files changed

+299
-0
lines changed

‎Book/TheConstructionStudies/Chapter11-DualModeExecutionParity.md‎

Lines changed: 296 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,296 @@
1+
# Chapter 11: Dual-Mode Execution Parity
2+
3+
## What This Chapter Is
4+
5+
ARO programs can run in two modes: interpreted (`aro run`) and compiled (`aro build`). In theory, they should produce identical results. In practice, they share most infrastructure but diverge in subtle ways that are hard to detect until tests fail silently.
6+
7+
This chapter documents the sources of divergence, the systematic fixes applied, and the architectural patterns that prevent future drift.
8+
9+
---
10+
11+
## The Divergence Problem
12+
13+
The interpreter and binary paths share `ActionRegistry.shared`, `RuntimeContext`, and `EventBus.shared`. But two key subsystems have entirely separate implementations:
14+
15+
**Event dispatch**: The interpreter uses Swift typed events (`FileCreatedEvent`, `StateTransitionEvent`, etc.) routed through `EventBus.subscribe(to:)`. The compiled binary uses `DomainEvent` (an event type string plus a `[String: any Sendable]` payload dictionary) routed through `aro_runtime_register_handler`.
16+
17+
**Expression evaluation**: The interpreter evaluates expressions in `ExpressionEvaluator.swift` (Swift). The compiled binary evaluates them in `evaluateBinaryOp()` in `RuntimeBridge.swift` (also Swift, but a separate implementation with different behavior for edge cases).
18+
19+
This separation is necessary — the compiled binary cannot execute arbitrary Swift closures — but it creates a gap that widens every time a new feature is added to only one path.
20+
21+
---
22+
23+
## Source of Divergence 1: Verb Sets
24+
25+
### The Problem
26+
27+
`FeatureSetExecutor.executeAROStatement()` classifies verbs into sets to decide whether a statement needs execution or can be skipped. These sets were defined inline as local `let` declarations:
28+
29+
```swift
30+
// Before — local to FeatureSetExecutor (lines 304–317)
31+
let testVerbs: Set<String> = ["then", "assert"]
32+
let requestVerbs: Set<String> = ["call", "invoke"]
33+
let updateVerbs: Set<String> = ["update", "modify", "change", "set"]
34+
// ... 7 more sets
35+
```
36+
37+
Because these were local, any code that needed to classify verbs elsewhere had to duplicate the sets or remain inconsistent.
38+
39+
### The Fix
40+
41+
`Sources/ARORuntime/Core/VerbSets.swift` extracts the sets into a shared public enum:
42+
43+
```swift
44+
public enum VerbSets {
45+
public static let testVerbs: Set<String> = ["then", "assert"]
46+
public static let requestVerbs: Set<String> = ["call", "invoke"]
47+
public static let updateVerbs: Set<String> = ["update", "modify", "change", "set"]
48+
public static let createVerbs: Set<String> = ["create", "make", "build", "construct"]
49+
public static let mergeVerbs: Set<String> = ["merge", "combine", "join", "concat"]
50+
public static let computeVerbs: Set<String> = ["compute", "calculate", "derive"]
51+
public static let extractVerbs: Set<String> = ["extract", "parse", "get"]
52+
public static let queryVerbs: Set<String> = ["filter", "map", "reduce", "aggregate", "split"]
53+
public static let responseVerbs: Set<String> = ["write", "read", "store", "save", "persist",
54+
"log", "print", "send", "emit", "notify",
55+
"alert", "signal", "broadcast"]
56+
public static let serverVerbs: Set<String> = ["start", "stop", "restart", "keepalive",
57+
"schedule", "stream", "subscribe",
58+
"sleep", "delay", "pause"]
59+
}
60+
```
61+
62+
`FeatureSetExecutor` now references these via `VerbSets.testVerbs` etc. Any future code that classifies verbs has a single authoritative source.
63+
64+
---
65+
66+
## Source of Divergence 2: Integer Division
67+
68+
### The Problem
69+
70+
Integer division produced different results in the two modes.
71+
72+
**Interpreter** (`ExpressionEvaluator.swift`, `.divide` case):
73+
74+
```swift
75+
// Before fix — always returned Double via numericOperation
76+
case .divide:
77+
return try numericOperation(left, right) { $0 / $1 }
78+
```
79+
80+
This meant `7 / 2` evaluated to `3.5` in interpreter mode.
81+
82+
**Binary** (`RuntimeBridge.swift`, `evaluateBinaryOp()`):
83+
84+
```swift
85+
case "/":
86+
if let li = left as? Int, let ri = right as? Int {
87+
guard ri != 0 else { return 0 }
88+
return li / ri // Integer floor division → 3
89+
}
90+
// ... fallback to double
91+
```
92+
93+
### The Fix
94+
95+
The interpreter now matches the binary behavior: Int/Int returns Int.
96+
97+
```swift
98+
case .divide:
99+
// Int/Int → integer floor division (matches binary mode evaluateBinaryOp behavior)
100+
if let li = left as? Int, let ri = right as? Int {
101+
guard ri != 0 else { return 0 }
102+
return li / ri
103+
}
104+
return try numericOperation(left, right) { $0 / $1 }
105+
```
106+
107+
This is a behavioral change: ARO integer division now truncates toward zero, consistent with most languages.
108+
109+
---
110+
111+
## Source of Divergence 3: Event Handler Registration
112+
113+
### The Architecture
114+
115+
The interpreter registers event handlers during program startup by subscribing Swift closures to typed events:
116+
117+
```swift
118+
// Interpreter — ExecutionEngine.registerNotificationEventHandlers
119+
eventBus.subscribe(to: NotificationSentEvent.self) { event in
120+
// evaluate when condition, then execute feature set
121+
}
122+
```
123+
124+
The compiled binary cannot use Swift closures at the C ABI boundary. Instead, `LLVMCodeGenerator` emits calls to C-callable registration functions at program startup, passing a function pointer to the compiled feature set:
125+
126+
```
127+
// Generated LLVM IR (pseudocode)
128+
call void @aro_runtime_register_notification_handler(
129+
runtime_ptr,
130+
handler_func_ptr,
131+
when_condition_json_ptr
132+
)
133+
```
134+
135+
### The DomainEvent Co-Publishing Pattern
136+
137+
For the binary path to receive events, every action that fires a typed event must also publish a `DomainEvent` to `EventBus.shared`. The `registerCompiledHandler` function in `RuntimeBridge.swift` subscribes to these `DomainEvents` and calls the compiled handler function.
138+
139+
**Pattern** (applies to all event-generating actions):
140+
141+
```swift
142+
// 1. Publish typed event for interpreter handlers
143+
if let eventBus = context.eventBus {
144+
await eventBus.publishAndTrack(MyTypedEvent(/* ... */))
145+
} else {
146+
context.emit(MyTypedEvent(/* ... */))
147+
}
148+
149+
// 2. Co-publish DomainEvent for binary mode handlers
150+
// DomainEvent eventType: "MyEventType"
151+
// DomainEvent payload: { "key1": Value, "key2": Value, ... }
152+
EventBus.shared.publish(DomainEvent(eventType: "MyEventType", payload: [
153+
"key1": value1,
154+
"key2": value2
155+
]))
156+
```
157+
158+
### Payload Schemas
159+
160+
Each event type has a defined payload schema. These are documented in comments at each callsite:
161+
162+
| Event Type | Payload Keys |
163+
|------------|--------------|
164+
| `StateTransition` | `fromState: String`, `toState: String`, `fieldName: String`, `objectName: String`, `entityId: String?` |
165+
| `NotificationSent` | `message: String`, `target: String`, `user: targetObj`, `[targetName]: targetObj`, plus all target object fields spread at top level |
166+
| `file.created` / `file.modified` / `file.deleted` | `path: String` |
167+
| `websocket.connected` | `connectionId: String`, `path: String`, `remoteAddress: String` |
168+
| `websocket.disconnected` | `connectionId: String`, `reason: String` |
169+
| `websocket.message` | `connectionId: String`, `message: String` |
170+
| `socket.connected` | `connection: { id: String, remoteAddress: String }` |
171+
| `socket.data` | `packet: { message: String, buffer: String, data: String, connection: String }` |
172+
| `socket.disconnected` | `event: { connectionId: String, reason: String }` |
173+
| `KeyPress` | `key: String` |
174+
175+
---
176+
177+
## The Handler Registration Pattern
178+
179+
Every new event type requires a corresponding `aro_runtime_register_*` function. All follow this template in `RuntimeBridge.swift`:
180+
181+
```swift
182+
@_cdecl("aro_runtime_register_my_event_handler")
183+
public func aro_runtime_register_my_event_handler(
184+
_ runtimePtr: UnsafeMutableRawPointer?, // runtime handle
185+
_ guardParamPtr: UnsafePointer<CChar>?, // optional guard (nullable)
186+
_ handlerFuncPtr: UnsafeMutableRawPointer? // compiled function pointer
187+
) {
188+
guard let runtimePtr, let handlerFuncPtr else { return }
189+
let runtimeHandle = Unmanaged<AROCRuntimeHandle>.fromOpaque(runtimePtr).takeUnretainedValue()
190+
let handlerAddress = Int(bitPattern: handlerFuncPtr)
191+
192+
runtimeHandle.runtime.registerCompiledHandler(
193+
eventType: "MyEventType",
194+
handlerName: "My Handler"
195+
) { @Sendable event in
196+
// 1. Evaluate guard condition (if any)
197+
// 2. Run handler on pthread (NOT GCD — avoids 64-thread limit)
198+
await withCheckedContinuation { continuation in
199+
Thread {
200+
let contextHandle = AROCContextHandle(runtime: runtimeHandle,
201+
featureSetName: "My Handler")
202+
// Bind event payload to context
203+
contextHandle.context.bind("event", value: event.payload)
204+
for (k, v) in event.payload {
205+
contextHandle.context.bind("event:\(k)", value: v)
206+
}
207+
// Execute compiled handler
208+
let handlerFunc = unsafeBitCast(/* reconstructed ptr */,
209+
to: HandlerFunc.self)
210+
let result = handlerFunc(contextPtr)
211+
// Cleanup
212+
continuation.resume()
213+
}.start()
214+
}
215+
}
216+
}
217+
```
218+
219+
**Why pthreads, not GCD?** GCD's cooperative thread pool has a 64-thread limit. During intensive event processing (many events firing handlers concurrently), GCD deadlocks when all 64 threads are blocked waiting for continuation resumes. Foundation `Thread` bypasses this limit. The `CompiledExecutionPool.shared` semaphore prevents unbounded thread creation.
220+
221+
The three-step pattern for every new event handler:
222+
223+
1. **`LLVMExternalDeclEmitter.swift`**: Declare the C function with LLVM types
224+
2. **`LLVMCodeGenerator.registerEventHandlers`**: Detect the business activity pattern and emit the registration call
225+
3. **`RuntimeBridge.swift`**: Implement the `@_cdecl` function
226+
227+
---
228+
229+
## The `when` Guard: Interpreter vs Binary
230+
231+
Handler feature sets can have a `when` guard:
232+
233+
```aro
234+
(Greet User: NotificationSent Handler) when <age> >= 16 {
235+
(* ... *)
236+
}
237+
```
238+
239+
**Interpreter**: `ExecutionEngine` evaluates this expression inline using `ExpressionEvaluator` with the target object's fields bound to context.
240+
241+
**Binary**: `LLVMCodeGenerator` serializes the `whenCondition` AST node to JSON using `serializeExpression()`:
242+
243+
```json
244+
{"$binary":{"op":">=","left":{"$var":"age"},"right":{"$literal":16}}}
245+
```
246+
247+
This JSON is passed as a string constant to the registration function. At runtime, `evaluateExpressionJSON()` in `RuntimeBridge.swift` deserializes and evaluates it against a `RuntimeContext` populated with the event payload.
248+
249+
This means the binary `when` guard evaluates against a flat payload dictionary, so the payload must spread the target object's fields at top level.
250+
251+
---
252+
253+
## Test Coverage: The mode: both Directive
254+
255+
Every `test.hint` file has a `mode` field:
256+
257+
| Value | Meaning |
258+
|-------|---------|
259+
| `both` | Run in interpreter and compiled binary modes, compare output |
260+
| `interpreter` | Run interpreter only (binary mode unsupported) |
261+
262+
Out of 85 examples, **81 currently run in `mode: both`** (including the default, which is `both`). The 4 interpreter-only examples have open issues:
263+
264+
| Example | Issue | Root Cause |
265+
|---------|-------|------------|
266+
| `SocketClient` | #134 | `AROSocketClient` uses `ManagedAtomic<Bool>` → SIGSEGV in binary |
267+
| `MultiService` | #134 | Depends on SocketClient fix |
268+
| `Scoping` | #135 | `AppReady Handler` event payload structure differs in binary mode |
269+
| `EventReplay` | #136 | `EventRecorder.swift` not implemented in C bridge |
270+
271+
The `occurrence-check: true` hint enables order-independent output comparison, which is essential for event handlers that fire asynchronously in binary mode.
272+
273+
---
274+
275+
## Verification Checklist for New Event Types
276+
277+
When adding a new action that fires events:
278+
279+
1. **Add DomainEvent co-publish** after the typed event publish
280+
2. **Document the payload schema** with a `// DomainEvent eventType: payload:` comment
281+
3. **Add `@_cdecl` registration function** in `RuntimeBridge.swift`
282+
4. **Declare the extern** in `LLVMExternalDeclEmitter.swift`
283+
5. **Detect the business activity** in `LLVMCodeGenerator.registerEventHandlers` (before generic `hasSuffix(" Handler")`)
284+
6. **Spread guard fields** into the DomainEvent payload if the handler has a `when` condition
285+
7. **Add or update an example** with `mode: both` and `occurrence-check: true`
286+
8. **Run** `swift build -c release && ./test-examples.pl`
287+
288+
---
289+
290+
## Lessons
291+
292+
**Silent divergence is the worst kind of bug.** A binary that produces wrong results without crashing is harder to diagnose than one that crashes immediately. The `mode: both` test directive is the primary defense: any behavioral difference between interpreter and binary becomes a test failure.
293+
294+
**Co-publishing is cheaper than unification.** A clean architectural solution would use a single event system for both modes. In practice, the typed event system is deeply integrated with the interpreter (closures, `async/await`, `publishAndTrack`), while the binary needs C-callable, pthread-compatible registration. DomainEvent co-publishing bridges the two worlds with minimal coupling and no breaking changes.
295+
296+
**Payload schemas are contracts.** The `// DomainEvent payload:` comments are not just documentation — they define the interface between the action that fires the event and the `RuntimeBridge` function that receives it. When the payload changes, both sides must be updated atomically.

‎Book/TheConstructionStudies/STRUCTURE.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,9 @@ Swift-C-LLVM interoperability. @_cdecl functions. Handle management. Descriptor
4141
### [Chapter 10: Critical Assessment](Chapter10-CriticalAssessment.md)
4242
What works well. What doesn't work. Design decisions we'd reconsider. Lessons for language implementers.
4343

44+
### [Chapter 11: Dual-Mode Execution Parity](Chapter11-DualModeExecutionParity.md)
45+
Sources of interpreter/binary divergence. VerbSets shared module. Integer division parity. DomainEvent co-publishing pattern. Handler registration template. Payload schema contracts. The `mode: both` test directive.
46+
4447
## Appendices
4548

4649
### [Appendix A: Source Map](Appendix-SourceMap.md)

0 commit comments

Comments
 (0)