Skip to content

Test fixes#9

Merged
hawkeyexl merged 10 commits intomainfrom
test-fixes
Nov 22, 2025
Merged

Test fixes#9
hawkeyexl merged 10 commits intomainfrom
test-fixes

Conversation

@hawkeyexl
Copy link
Contributor

@hawkeyexl hawkeyexl commented Nov 22, 2025

Summary by CodeRabbit

  • Bug Fixes

    • Improved test-run reliability by preventing duplicate completion paths and race conditions in process shutdown, with clearer lifecycle logging for child processes.
  • Tests

    • Fixed a test description typo: "Run tests successfully."
  • Chores

    • CI workflow now triggers for all pull request events (no branch filtering).
    • Test artifact configuration: default log level set to info; browser/platform contexts consolidated (Firefox-specific contexts removed).

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 22, 2025

Walkthrough

Unified test-runner completion handling and Docker invocation tweaks in test/runTests.test.js; adjusted CI workflow trigger in .github/workflows/build-push.yml to run on all pull requests; simplified test artifact config in test/artifacts/config.json and lowered log level.

Changes

Cohort / File(s) Change Summary
Test runner guard & flow refactor
test/runTests.test.js
Fixed test description typo; added hasCompleted flag and handleCompletion guard to ensure single resolve/reject; increased Docker resource flags and maxBuffer; consolidated error/exit/close handling to a single close-based path that logs code/signal, reads/parses outputFile, asserts zero failures, performs cleanup, then resolves or rejects via the guard.
CI workflow trigger
.github/workflows/build-push.yml
Removed branch filtering under pull_request so the workflow triggers for all pull request events.
Test artifact configuration
test/artifacts/config.json
Changed logLevel from "debug" to "info"; collapsed per-platform contexts into a single Chrome context covering ["mac","linux","windows"]; removed Firefox-specific and separate Windows Chrome contexts.

Sequence Diagram(s)

sequenceDiagram
    participant Child as Child Process
    participant Handler as handleCompletion
    participant FS as File System
    participant Promise as Test Promise

    rect rgb(245,250,245)
    Note over Handler: hasCompleted = false\nhandleCompletion enforces single completion
    end

    alt Child emits error
        Child->>Handler: error(err)
        Handler->>Handler: if not hasCompleted\n set hasCompleted = true
        Handler->>Promise: reject(err)
    else Child closes/exits
        Child->>Handler: close(code, signal)
        Handler->>Handler: if not hasCompleted\n set hasCompleted = true
        Handler->>FS: read(outputFile)
        FS-->>Handler: file contents
        Handler->>Handler: parse results, assert failures === 0
        alt assertions pass
            Handler->>Promise: resolve(results)
        else
            Handler->>Promise: reject(assertionError)
        end
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Review focus:
    • hasCompleted guard correctness and race conditions across error/close handlers.
    • Docker invocation flags and maxBuffer sizing implications.
    • File-read/parsing error paths and cleanup in the unified close handler.
    • CI trigger change impact on workflow run volume and expected behavior.

Possibly related PRs

Poem

🐰 I hopped through logs and file reads bright,
One completion now ends the long night,
Docker trimmed and workflows set free,
A tidy run, from error to glee. 🥕

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'Test fixes' is vague and generic, using non-descriptive language that doesn't convey the specific nature of the changes made to the test suite and workflow. Consider using a more specific title that reflects the main changes, such as 'Fix test process handling and unify test configurations' or 'Improve test process termination handling and workflow triggers'.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch test-fixes

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 210960e and 50fb040.

📒 Files selected for processing (3)
  • .github/workflows/build-push.yml (0 hunks)
  • test/artifacts/config.json (2 hunks)
  • test/runTests.test.js (2 hunks)
💤 Files with no reviewable changes (1)
  • .github/workflows/build-push.yml
🧰 Additional context used
🧬 Code graph analysis (1)
test/runTests.test.js (1)
scripts/build.js (2)
  • version (8-8)
  • os (30-30)
🔇 Additional comments (3)
test/artifacts/config.json (1)

6-6: Config consolidation and log level change look good

Using logLevel: "info" and a single chrome headless context spanning ["mac", "linux", "windows"] is a sensible simplification for CI and aligns with a unified test path. No issues from a structure or intent standpoint.

Also applies to: 18-21

test/runTests.test.js (2)

60-67: Single-completion guard and close handler match the intended failure semantics

The hasCompleted/handleCompletion pattern cleanly enforces a single resolve/reject across error and close events, preventing double completion. The updated close handler correctly:

  • Accepts (code, signal) and logs both.
  • Rejects immediately when signal != null, before attempting to read the output file, with a clear error message containing the signal name.
  • Preserves the code !== null && code !== 0 check and returns after rejection.
  • Only on a clean exit, parses results.json, asserts zero failures, removes the file, and resolves.

This aligns well with the PR objectives for robust termination handling and result validation.

Also applies to: 81-84, 86-111


55-67: Revert: the async/Promise pattern is not a functional bug

When an async function returns a Promise, the async function's returned Promise "adopts" that promise—it is resolved/rejected with the same outcome. This adoption uses Promise resolution mechanics and the Promise settles on the microtask queue.

In your code, when async () => { return new Promise(...) } executes, the async function's returned Promise adopts the inner Promise; they settle together. Mocha will correctly wait for both.

The async keyword is stylistically redundant when manually creating and returning a Promise, but it is not a functional bug. The test will wait for completion as intended.

Likely an incorrect or invalid review comment.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
test/runTests.test.js (1)

59-90: hasExited guard correctly fixes double‑rejection; optionally improve exit diagnostics

The hasExited pattern around the "error" and "exit" handlers looks solid and should prevent the Promise from being rejected more than once, even if both events fire.

As an optional refinement, you might improve the "exit" handler’s diagnostics for cases where the process terminates via signal (i.e., code === null) instead of relying on the later JSON read to fail:

-      runTests.on("exit", (code) => {
+      runTests.on("exit", (code, signal) => {
         if (hasExited) return;
         hasExited = true;
-        
-        console.log(`Child process exited with code ${code}`);
-        
-        if (code !== null && code !== 0) {
-          reject(new Error(`Docker process exited with code ${code}`));
-          return;
-        }
+        
+        if (code === null) {
+          reject(new Error(`Docker process exited due to signal ${signal || "unknown"}`));
+          return;
+        }
+
+        console.log(`Child process exited with code ${code}`);
+
+        if (code !== 0) {
+          reject(new Error(`Docker process exited with code ${code}`));
+          return;
+        }

Not required for correctness, but it will produce clearer failures when Docker is killed by a signal rather than exiting with a numeric code.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between acf89c6 and e305a8c.

📒 Files selected for processing (1)
  • test/runTests.test.js (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: build (windows-latest)
  • GitHub Check: build (ubuntu-latest)

@hawkeyexl
Copy link
Contributor Author

@copilot Could you have a go at this? It worked locally.

Copy link
Contributor

Copilot AI commented Nov 22, 2025

@hawkeyexl I've opened a new pull request, #10, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot AI and others added 2 commits November 22, 2025 18:02
Co-authored-by: hawkeyexl <5209367+hawkeyexl@users.noreply.github.com>
Fix test race condition with close event
@hawkeyexl hawkeyexl changed the title test rixes Test fixes Nov 22, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e305a8c and 210960e.

📒 Files selected for processing (1)
  • test/runTests.test.js (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: build (windows-latest)
  • GitHub Check: build (ubuntu-latest)
🔇 Additional comments (3)
test/runTests.test.js (3)

55-55: LGTM: Typo fixed.

The test description typo has been corrected.


58-66: LGTM: Completion guard prevents double resolution.

The handleCompletion guard is a solid pattern that prevents race conditions between the error and close events, ensuring the promise resolves or rejects exactly once.


80-83: LGTM: Error handler properly guarded.

The error handler correctly uses handleCompletion to prevent double rejection.

@hawkeyexl
Copy link
Contributor Author

@copilot In test/runTests.test.js around lines 85 to 106, the close handler only accepts
the exit code and ignores the signal, so when the child process is terminated by
a signal the logic continues as if it succeeded; update the close listener
signature to accept both (code, signal), then before attempting to read the
output file reject if signal is non-null (include the signal name in the error
message), and preserve the existing check to reject when code !== null && code
!== 0; ensure you return after reject to avoid further processing.

Also update the test workflow to run in any pr.

Copy link
Contributor

Copilot AI commented Nov 22, 2025

@hawkeyexl I've opened a new pull request, #11, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot AI and others added 5 commits November 22, 2025 19:27
Co-authored-by: hawkeyexl <5209367+hawkeyexl@users.noreply.github.com>
Co-authored-by: hawkeyexl <5209367+hawkeyexl@users.noreply.github.com>
Co-authored-by: hawkeyexl <5209367+hawkeyexl@users.noreply.github.com>
Co-authored-by: hawkeyexl <5209367+hawkeyexl@users.noreply.github.com>
Fix signal handling in test close handler, enable workflow for all PRs, and add resource limits
@hawkeyexl hawkeyexl merged commit 4a33997 into main Nov 22, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants