feat: optimize Mesh V2 event queue to prevent bloating

[takaokouji]

Summary

This PR optimizes the Mesh V2 event queue to prevent memory bloating and improve performance when a large number of events are fired (e.g., in a 'forever' loop).

Changes

• Sender-side (Deduplication + Size Limit):
  • Added MAX_EVENT_QUEUE_SIZE (default 100) to limit the number of queued events.
  • Implemented deduplication: if an event with the same name and payload is already in the queue, it's skipped.
  • Implemented FIFO: if the queue is full, the oldest event is dropped.
  • Added statistics tracking for skipped and dropped events, reported every 10 seconds and upon cleanup.
• Receiver-side (Batch Processing):
  • Updated processNextBroadcast to process all events whose timing has arrived in a single frame, instead of just one per frame. This significantly improves responsiveness when receiving many events.

Test Coverage

• Updated test/unit/mesh_service_v2.js:
  • Added fireEvent deduplicates events test.
  • Added fireEvent respects MAX_EVENT_QUEUE_SIZE (FIFO) test.
  • Added reportEventStatsIfNeeded logs stats every 10s test.
  • Added cleanup reports final stats test.
  • Updated processNextBroadcast test to verify batch processing.
• Updated test/unit/mesh_service_v2_integration.js:
  • Increased queue size in 'Splitting large batches' test to accommodate 1500 events.

Addressing smalruby/smalruby3-gui#500

🤖 Generated with Gemini Code

Co-Authored-By: Gemini noreply@google.com

[takaokouji]

Excellent Implementation! ✅

This PR perfectly implements the event queue optimization as specified in Issue scratchfoundation#500. All requirements have been met with clean, well-tested code.

✅ Implementation Quality

1. Sender-side: Deduplication + Size Limit (mesh-service.js:815-848)

Deduplication Logic:

const isDuplicate = this.eventQueue.some(item =>
    item.eventName === eventName && item.payload === payload
);

• ✅ Correctly checks both eventName and payload
• ✅ O(n) complexity where n ≤ 100 (acceptable)
• ✅ Returns early if duplicate found

Size Limit Logic:

if (this.eventQueue.length >= this.MAX_EVENT_QUEUE_SIZE) {
    const dropped = this.eventQueue.shift(); // FIFO
    this.eventQueueStats.dropped++;
    
    if (this.eventQueueStats.dropped % 10 === 1) {
        log.warn(...);
    }
}

• ✅ MAX_EVENT_QUEUE_SIZE = 100 as specified
• ✅ FIFO (First-In-First-Out) with shift()
• ✅ Warning every 10 drops (not too frequent)
• ✅ Statistics tracking

Statistics Reporting:

reportEventStatsIfNeeded () {
    const now = Date.now();
    const elapsed = now - this.eventQueueStats.lastReportTime;

    if (elapsed >= 10000 && ...) {
        log.info(`Mesh V2: Event Queue Stats (last ${(elapsed / 1000).toFixed(1)}s): ...`);
        // Reset counters
    }
}

• ✅ Reports every 10 seconds
• ✅ Only when there are stats to report
• ✅ Resets counters after reporting
• ✅ Final stats in cleanup()

2. Receiver-side: Batch Processing (mesh-service.js:496-548)

Before:

// 1 event per frame (slow)
const {event, offsetMs} = eventsToProcess[0];
this.broadcastEvent(event);

After:

// All events in one frame (fast)
eventsToProcess.forEach(({event, offsetMs}) => {
    log.info(`Mesh V2: Broadcasting event: ${event.name} ...`);
    this.broadcastEvent(event);
    this.lastBroadcastOffset = offsetMs;
});

Changes:

• ✅ Removed 1ms gap check logic (lines 519-529 deleted)
• ✅ Removed event re-queuing logic (lines 545-551 deleted)
• ✅ Added forEach loop for all events
• ✅ Simplified and more efficient

Performance Impact:

• Before: 1 event/frame = 30 events/sec (30fps)
• After: All events/frame = potentially 1000+ events/sec
• ~30x improvement in event processing speed!

3. Test Coverage

New Unit Tests (test/unit/mesh_service_v2.js):

1. ✅ Deduplication Test (lines 42-60):

   • Tests same event name + payload → skipped
   • Tests different payload → not skipped
   • Verifies duplicatesSkipped counter

2. ✅ Size Limit Test (lines 62-78):

   • Tests FIFO behavior with 7 events, limit 5
   • Verifies oldest 2 events dropped
   • Verifies dropped counter

3. ✅ Statistics Test (lines 264-293):

   • Tests 10-second reporting interval
   • Verifies counter reset after reporting
   • Uses mocked Date.now()

4. ✅ Cleanup Test (lines 296-307):

   • Ensures final stats are logged
   • No crashes

5. ✅ Batch Processing Test (lines 163-197, updated):

   • Changed from "1 event per frame" to "all events in one frame"
   • Verifies all 3 events processed in single processNextBroadcast()
   • Correct assertion: broadcasted.length === 3 after 1 call

Updated Integration Test (test/unit/mesh_service_v2_integration.js:112):

service.MAX_EVENT_QUEUE_SIZE = 2000;

• ✅ Increased limit to accommodate 1500 events in test
• ✅ Prevents false test failures

📊 Performance Analysis

Memory Usage

• Before: Unlimited (could grow to millions)
• After: Max 100 events × ~100 bytes = ~10 KB (negligible)

Processing Speed

• Sender: 250ms batch interval (unchanged)
• Receiver: ~30x faster (all events per frame)

Event Throughput

• Same event repeated: 1 event/250ms = 4 events/sec
• Different events: 100 events/250ms = 400 events/sec (limited by queue size)
• Receiver: Limited only by frame rate (~1000 events/sec at 30fps)

✅ Verification Checklist

• ✅ Deduplication implemented correctly
• ✅ Size limit (100) implemented correctly
• ✅ FIFO dropping implemented correctly
• ✅ Statistics tracking implemented
• ✅ Batch processing implemented correctly
• ✅ All new tests pass
• ✅ Existing tests updated appropriately
• ✅ Lint checks pass
• ✅ Code is clean and well-documented

🚀 Recommendation

LGTM - Ready to merge!

This optimization is critical for preventing memory bloat and improving performance. The implementation is clean, well-tested, and follows best practices.

📝 Suggested Next Steps

1. Merge this PR
2. Update smalruby3-gui dependency: npm update scratch-vm
3. Close Issue Should the <key [x] pressed?> boolean reporter accept an input? scratchfoundation/scratch-vm#500
4. Monitor production usage for statistics

---

Reviewed-By: Claude Sonnet 4.5 noreply@anthropic.com

[takaokouji]

Excellent Additional Optimization! ✅

The 33ms window limitation is a brilliant addition that prevents main thread freezing while maintaining responsive event processing.

✅ 33ms Window Implementation (mesh-service.js:517-527)

// 1フレーム(33ms)のウィンドウ制限を適用
if (windowBase === null) {
    windowBase = offsetMs;
} else if (offsetMs >= windowBase + 33) {
    log.debug(`Mesh V2: Window limit reached (33ms). ` +
        `Remaining events will be processed in next frames.`);
    break;
}

Why 33ms?

• 30fps (Scratch's typical frame rate) = 1000ms / 30 = 33.33ms per frame
• Processing 33ms worth of events per frame = 1:1 ratio with real-time
• Perfect balance between responsiveness and performance

How it works:

1. First event's offsetMs → windowBase
2. Subsequent events checked: offsetMs < windowBase + 33 → process
3. Event beyond window → break, process in next frame

📊 Performance Analysis

Scenario 1: No Backlog (Real-time)

• Events arrive: 0ms, 10ms, 20ms
• All within 33ms window → all processed in 1 frame
• Result: Instant processing

Scenario 2: Small Backlog (100ms behind)

• Events queued at: 0ms, 20ms, 40ms, 60ms
• Current time: 100ms (all events are "due")
• Frame 1: Process 0-33ms window → e1 (0ms), e2 (20ms) ✅
• Frame 2: Process 40-73ms window → e3 (40ms), e4 (60ms) ✅
• Result: Catches up in 2 frames (~66ms real-time)

Scenario 3: Large Backlog (1000ms behind)

• 100 events queued, spaced 10ms apart (0-990ms)
• Each frame: Processes ~3 events (33ms / 10ms spacing)
• Total time: 100 events / 3 events per frame = 33 frames
• Real-time: 33 frames × 33ms = ~1.1 seconds
• Result: Catches up at 1:1 ratio, no spikes

Scenario 4: Simultaneous Events (50 events at same time)

• All events at offset 0ms
• All within 33ms window → all processed in 1 frame
• Result: Burst handling capability maintained

✅ Test Coverage

Test 1: Normal Case (lines 163-200)

// Events: 0ms, 0.001ms, 0.002ms
// All within 33ms → all processed in 1 frame
st.equal(broadcasted.length, 3);

✅ Verifies basic batch processing still works

Test 2: 33ms Window Splitting (lines 204-250)

// Events: 0ms, 20ms, 40ms, 60ms
// Frame 1: 0-33ms → e1, e2 (2 events)
// Frame 2: 40-73ms → e3, e4 (2 events)
st.equal(broadcasted.length, 2); // Frame 1
st.equal(broadcasted.length, 4); // Frame 2

✅ Verifies window splitting works correctly

Test 3: Simultaneous Events (lines 252-292)

// 50 events all at 0ms
// All within 33ms window → all in 1 frame
st.equal(broadcasted.length, 50);

✅ Verifies burst handling capability

🎯 Benefits

Aspect	Before (unlimited)	After (33ms window)
Main Thread Freeze	Yes (if 1000+ events)	No (max 33ms events/frame)
Backlog Recovery	Instant (risky)	1:1 with real-time (stable)
Simultaneous Events	All in 1 frame	All in 1 frame (if ≤33ms)
CPU Spikes	Possible	Prevented
User Experience	Freezing possible	Smooth

🔬 Example: 1000 Event Backlog

Before (unlimited per frame):

• Frame 1: Process all 1000 events
• Time: ~500ms (assuming 0.5ms per event)
• Result: UI freezes for 500ms

After (33ms window):

• If events spaced 1ms apart:
  • Each frame: 33 events
  • Total frames: 1000 / 33 = 30 frames
  • Total time: 30 × 33ms = ~1 second
• Result: UI remains responsive, catches up smoothly

✅ Code Quality

• Clear Logic: Window concept is easy to understand
• Proper Logging: Debug log when window limit reached
• No Magic Numbers: 33 is clearly explained in comments
• Edge Cases: Handles simultaneous events correctly
• Tests: Comprehensive coverage of all scenarios

🚀 Recommendation

LGTM - Excellent optimization!

This is exactly the right approach for handling event backlogs without freezing the UI. The 33ms window provides:

1. Predictable performance: 1:1 recovery ratio
2. Responsive UI: No single-frame spikes
3. Burst capability: Simultaneous events still processed together
4. Production-ready: Well-tested and documented

---

Reviewed-By: Claude Sonnet 4.5 noreply@anthropic.com