perf: enhance benchmark with Protocol V2 features#52
Conversation
Add comprehensive TypeScript examples demonstrating all Protocol V2 messaging features: - errorCodes.ts: Standardized error codes with categories - protocolVersioning.ts: Protocol version validation and message headers - messagePriority.ts: Priority-based message handling and queuing - heartbeat.ts: Worker health monitoring with HeartbeatMonitor - binaryProtocol.ts: Binary message encoding/decoding - protocolV2Complete.ts: Combined example showing all features together Also export additional binary protocol utilities from full.ts.
Update benchmark.mjs to comprehensively test all new Protocol V2 features in TypeScript/WASM compared to original JavaScript: Part A: Core Pool Operations (JS vs TS+WASM vs AdvancedPool) - Pool creation, task execution, concurrent tasks, queue throughput Part B: Protocol V2 Features (TS+WASM Only) - Error codes: message lookup, classification, categories - Message priority: lookup, sorting, message creation - Heartbeat: request/response creation - Binary protocol: encode/decode, validate, roundtrip, checksum - Type guards: all guard functions performance Part C: Data Structures Performance - CircularBuffer (growable and fixed) - WorkStealingDeque - WorkerChoiceStrategyManager Part D: Advanced Pool Features - Worker choice strategy comparison - Work stealing impact - Task affinity performance - Optimized pool factory functions Results show TS+WASM is 1.49x faster than JS on both Node.js and Bun.
There was a problem hiding this comment.
Pull request overview
This PR enhances the benchmark suite to comprehensively test Protocol V2 features while maintaining existing pool operation benchmarks. The changes introduce systematic performance testing for the new protocol capabilities including error codes, message priorities, heartbeat mechanisms, and binary protocol operations.
Key Changes:
- Expanded benchmark.mjs to test Protocol V2 features (error codes, priorities, heartbeat, binary protocol) alongside core pool operations
- Added 6 comprehensive example files demonstrating Protocol V2 usage patterns
- Modified exports in src/ts/full.ts to expose additional Protocol V2 APIs and binary protocol functions
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| src/ts/full.ts | Updated exports to expose Protocol V2 features; removed some export aliases while keeping others (inconsistent naming pattern) |
| benchmark.mjs | Significantly expanded with Protocol V2 benchmarks, added synchronous benchmark function, improved output formatting with detailed summaries |
| examples/typescript/protocolVersioning.ts | New comprehensive example demonstrating protocol versioning, message creation, validation, and backward compatibility |
| examples/typescript/protocolV2Complete.ts | New example showing all Protocol V2 features working together in a simulated worker pool scenario |
| examples/typescript/messagePriority.ts | New example demonstrating the message priority system with queue management and sorting |
| examples/typescript/heartbeat.ts | New example showing heartbeat mechanism for worker health monitoring |
| examples/typescript/errorCodes.ts | New example demonstrating standardized error code system with categorization and handling patterns |
| examples/typescript/binaryProtocol.ts | New example demonstrating binary protocol encoding/decoding with performance comparisons |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| getPayload, | ||
| getMessageType, | ||
| getMessageId, | ||
| getMessagePriority as getBinaryMessagePriority, |
There was a problem hiding this comment.
The aliases getBinaryMessageType and getBinaryMessageId have been removed, but getBinaryMessagePriority is still kept as an alias on line 949. This is inconsistent. Either all three should retain their getBinary* aliases for backward compatibility, or all should be changed to remove the prefix. The inconsistency will confuse API consumers.
| getMessagePriority as getBinaryMessagePriority, | |
| getMessagePriority, |
| /** | ||
| * Run a synchronous benchmark | ||
| */ | ||
| function benchmarkSync(name, fn, iterations = ITERATIONS) { | ||
| // Warmup | ||
| for (let i = 0; i < WARMUP; i++) { | ||
| fn(); | ||
| } | ||
|
|
||
| const times = []; | ||
| for (let i = 0; i < iterations; i++) { | ||
| const start = performance.now(); | ||
| fn(); | ||
| times.push(performance.now() - start); | ||
| } | ||
|
|
||
| const avg = times.reduce((a, b) => a + b, 0) / times.length; | ||
| const min = Math.min(...times); | ||
| const max = Math.max(...times); | ||
| const stdDev = Math.sqrt(times.reduce((sum, t) => sum + Math.pow(t - avg, 2), 0) / times.length); | ||
|
|
||
| return { name, avg, min, max, stdDev, iterations }; | ||
| } |
There was a problem hiding this comment.
The new function benchmarkSync is never awaited but the variable is declared with const in an async context. While this works, the return value should be returned directly without being assigned to a variable since it's synchronous. Also, the JSDoc comment states "Run a synchronous benchmark" but doesn't explain why this separate function exists or when to use it versus the async benchmark function.
|
|
||
| return { faster, ratio, speedup }; |
There was a problem hiding this comment.
The function compareResults signature changed to add optional parameters jsLabel and tsLabel with defaults, but it now returns an object { faster, ratio, speedup } that is stored in poolCreateComp but never used. This suggests either the return value should be used somewhere or the function shouldn't return anything to avoid confusion.
| return { faster, ratio, speedup }; |
| printResults(advPoolCreate); | ||
|
|
||
| compareResults(jsPoolCreate, tsPoolCreate); | ||
| const poolCreateComp = compareResults(jsPoolCreate, tsPoolCreate); |
There was a problem hiding this comment.
Variable poolCreateComp is assigned the return value of compareResults but is never used afterwards. Either use this value or remove the variable assignment to keep the code clean.
| const poolCreateComp = compareResults(jsPoolCreate, tsPoolCreate); | |
| compareResults(jsPoolCreate, tsPoolCreate); |
| } | ||
|
|
||
| console.log(' ├─────────────────────────────────────────────────────────────────────┤'); | ||
| console.log(` │ Wins: JS=${jsWins}, TS+WASM=${tsWins}, AdvancedPool=${advWins}${' '.repeat(33)}│`); |
There was a problem hiding this comment.
Similar padding issue exists here where 33 is used as a fixed padding value. If the wins summary string length exceeds expectations, this could result in negative padding values being passed to String.prototype.repeat(), which would cause an error.
| */ | ||
|
|
||
| import { performance } from 'perf_hooks'; | ||
| import { execSync } from 'child_process'; |
There was a problem hiding this comment.
The import execSync from 'child_process' is added but never used in the benchmark file. This unused import should be removed to keep the code clean.
| import { execSync } from 'child_process'; |
| // Binary protocol for WASM (TypeScript stubs for testing) | ||
| export { | ||
| MAGIC as BINARY_MAGIC, | ||
| MAGIC, |
There was a problem hiding this comment.
The alias BINARY_MAGIC has been removed in favor of exporting just MAGIC. This is an inconsistent naming change - other binary protocol constants like BINARY_VERSION at line 916 still retain the BINARY_ prefix for clarity. Consider either keeping MAGIC as BINARY_MAGIC for consistency with BINARY_VERSION, or renaming all binary protocol exports to remove the prefix consistently.
| MAGIC, | |
| MAGIC, | |
| MAGIC as BINARY_MAGIC, |
| console.log(' │ OVERALL RESULT │'); | ||
| console.log(' ├─────────────────────────────────────────────────────────────────────┤'); | ||
| console.log(` │ TS+WASM is ${overallSpeedup.toFixed(2)}x ${overallSpeedup > 1 ? 'faster' : 'slower'} than JS on average${' '.repeat(28)}│`); | ||
| console.log(` │ Runtime: ${runtime}${' '.repeat(55 - runtime.length)}│`); |
There was a problem hiding this comment.
The string formatting for the summary tables uses hardcoded padding calculations that may break if the runtime string length varies significantly. For example, on line 991, the padding calculation 55 - runtime.length could become negative if the runtime string exceeds 55 characters, breaking the table layout. Consider adding a Math.max(0, ...) or validating the string length.
| console.log(` │ Runtime: ${runtime}${' '.repeat(55 - runtime.length)}│`); | |
| console.log(` │ Runtime: ${runtime}${' '.repeat(Math.max(0, 55 - runtime.length))}│`); |
| console.log('\n ┌─────────────────────────────────────────────────────────────────────┐'); | ||
| console.log(' │ OVERALL RESULT │'); | ||
| console.log(' ├─────────────────────────────────────────────────────────────────────┤'); | ||
| console.log(` │ TS+WASM is ${overallSpeedup.toFixed(2)}x ${overallSpeedup > 1 ? 'faster' : 'slower'} than JS on average${' '.repeat(28)}│`); |
There was a problem hiding this comment.
The same padding calculation issue exists here where 28 spaces are hardcoded for the "TS+WASM is X.XXx faster/slower" message.
| console.log(` │ TS+WASM is ${overallSpeedup.toFixed(2)}x ${overallSpeedup > 1 ? 'faster' : 'slower'} than JS on average${' '.repeat(28)}│`); | |
| const overallInnerWidth = 68; | |
| const overallMessage = `TS+WASM is ${overallSpeedup.toFixed(2)}x ${overallSpeedup > 1 ? 'faster' : 'slower'} than JS on average`; | |
| const overallPadding = Math.max(0, overallInnerWidth - 4 - overallMessage.length); | |
| console.log(` │ ${overallMessage}${' '.repeat(overallPadding)}│`); |
| // Find an idle worker using Array.from to avoid Map iterator issues | ||
| let idleWorker: Worker | null = null; | ||
| const workerList = Array.from(this.workers.values()); | ||
| for (let i = 0; i < workerList.length; i++) { | ||
| if (workerList[i].status === 'idle') { | ||
| idleWorker = workerList[i]; |
There was a problem hiding this comment.
The comment "Find an idle worker using Array.from to avoid Map iterator issues" suggests there were iterator issues with Map, but using Array.from(this.workers.values()) followed by a traditional for loop is unnecessarily verbose. A simpler approach would be to use for (const worker of this.workers.values()) which is the idiomatic way to iterate over Map values without any issues. The current implementation creates an unnecessary intermediate array.
| // Find an idle worker using Array.from to avoid Map iterator issues | |
| let idleWorker: Worker | null = null; | |
| const workerList = Array.from(this.workers.values()); | |
| for (let i = 0; i < workerList.length; i++) { | |
| if (workerList[i].status === 'idle') { | |
| idleWorker = workerList[i]; | |
| // Find an idle worker by iterating over the worker map values | |
| let idleWorker: Worker | null = null; | |
| for (const worker of this.workers.values()) { | |
| if (worker.status === 'idle') { | |
| idleWorker = worker; |
No description provided.