test: skip daemon tests when heidid missing by heidi-dang · Pull Request #79 · heidi-dang/heidi-engine

heidi-dang · 2026-02-20T12:21:53Z

CI runners do not build the C++ daemon. Skip daemon tests at module import when heidid is unavailable to keep matrix deterministic.

* test: skip heidi_cpp-dependent tests when extension missing * chore: retrigger CI * fix: repair validate_clean smoke and relax JSON error assertion * Fix validate_schema error, update test regex, and resolve trusted path issue in 04_train_qlora.py. --------- Co-authored-by: Heidi <heidi@example.com>

gemini-code-assist · 2026-02-20T12:22:14Z

Summary of Changes

Hello @heidi-dang, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the stability and predictability of the test suite, particularly in continuous integration environments where the C++ daemon might not be present. It ensures that tests dependent on the daemon are gracefully skipped if the necessary binary is missing, preventing unnecessary failures. Additionally, it refines error handling within core utility functions to align with standard Python exception practices, making the codebase more robust and easier to debug.

Highlights

Test Suite Robustness: Implemented a mechanism to conditionally skip daemon-related tests when the heidid C++ binary is not available, ensuring deterministic CI outcomes.
Error Handling Improvement: Refactored JSONL utility functions to raise ValueError exceptions instead of calling sys.exit(1) for schema validation and JSON parsing errors, improving Pythonic error management.
Test Refactoring: Centralized the pytest.importorskip logic for heidi_cpp into a dedicated module and applied it across multiple test files, reducing redundancy.
Daemon Test Enhancements: Improved the test_daemon.py suite by dynamically locating the heidid binary and enhancing the daemon startup polling for greater reliability.

Changelog

.local/ml/scripts/split_holdout.py
- Updated the default output path for training data from .local to ./verified.
heidi_engine/utils/io_jsonl.py
- Reformatted the REQUIRED_KEYS set for improved readability.
- Replaced sys.exit(1) with raise ValueError for schema validation and JSON parsing errors in load_jsonl.
scripts/02_validate_clean.py
- Applied minor formatting adjustments to comments and function calls.
- Changed a function call from validate_schema to validate_semantic.
tests/_requires_heidi_cpp.py
- Added a new module to provide a pytest.importorskip mechanism for heidi_cpp, allowing tests to be skipped if the C++ extension is not available.
tests/test_budget_guardrails.py
- Modified to use pytest.importorskip("heidi_cpp") to conditionally run tests requiring the C++ daemon.
tests/test_core_integration.py
- Modified to use pytest.importorskip("heidi_cpp") to conditionally run tests requiring the C++ daemon.
tests/test_daemon.py
- Refactored to dynamically locate the heidid binary and skip tests if it's not found.
- Improved the daemon startup polling mechanism for reliability.
- Updated imports and added Path for path manipulation.
tests/test_doctor.py
- Modified to use pytest.importorskip("heidi_cpp") to conditionally run tests requiring the C++ daemon.
tests/test_hpo.py
- Modified to use pytest.importorskip("heidi_cpp") to conditionally run tests requiring the C++ daemon.
tests/test_jsonl_utils.py
- Added a blank line for formatting.
- Updated test data for test_save_and_load_roundtrip, test_save_creates_directories, and test_load_skips_blank_lines to include all REQUIRED_KEYS for journal validation.
- Changed test_load_handles_invalid_json to assert ValueError instead of SystemExit.
tests/test_loop_runner.py
- Modified to use pytest.importorskip("heidi_cpp") to conditionally run tests requiring the C++ daemon.
tests/test_perf_baseline.py
- Modified to use pytest.importorskip("heidi_cpp") to conditionally run tests requiring the C++ daemon.
tests/test_schema_lock.py
- Added a blank line for formatting.
- Changed error handling from SystemExit to ValueError for schema validation failures in load_jsonl.
- Updated assertions to check for ValueError messages.
tests/test_sec_redteam.py
- Modified to use pytest.importorskip("heidi_cpp") to conditionally run tests requiring the C++ daemon.

Activity

The pull request was created by heidi-dang to address issues with daemon tests failing in CI environments where the C++ daemon is not built.
The author implemented a strategy to skip these tests gracefully when the heidid binary is unavailable, ensuring a more stable and deterministic test matrix.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

The pull request introduces logic to skip daemon-related tests when required C++ components are missing, which is a good improvement for CI environments. No security vulnerabilities were found. However, there are a few critical issues: a missing import in the daemon tests will cause them to fail, and a logic error in the validation script incorrectly unpacks a return value, effectively bypassing a validation step and introducing redundant code execution.

gemini-code-assist · 2026-02-20T12:24:45Z

tests/test_daemon.py

+import os
+import shutil
+import socket
 import subprocess
 import time
-import urllib.request
+from pathlib import Path
+import pytest
 import json


The urllib.request import was removed in this change, but the module is still used in test_daemon_status_json (line 69) and test_daemon_train_now_action (line 84). This will result in a NameError when these tests are executed.

Suggested change

import os

import shutil

import socket

import subprocess

import time

import urllib.request

from pathlib import Path

import pytest

import json

import os

import shutil

import socket

import subprocess

import time

import urllib.request

from pathlib import Path

import pytest

import json

gemini-code-assist · 2026-02-20T12:24:45Z

scripts/02_validate_clean.py

    # Step 1: Schema validation
-    valid, reason = validate_schema(sample)
+    valid, reason = validate_semantic(sample)
    if not valid:
        return None, f"schema: {reason}"


There are multiple issues with this block:

Incorrect Unpacking: validate_semantic returns a nested tuple ((bool, str), dict). Unpacking it into valid, reason sets valid to the tuple (bool, str), which will always evaluate to True in a boolean context, effectively bypassing the validation check.

Redundancy: validate_semantic is called again on line 392, making this call redundant.

Safety: This call is not guarded by HAS_SEMANTIC_VALIDATOR, which will cause a NameError if the validator failed to import.

Since semantic validation is handled later in the function (lines 391-399), this block should be removed.

gemini-code-assist · 2026-02-20T12:24:46Z

scripts/02_validate_clean.py

    return parser.parse_args()

-
    return True, "ok"


This return statement is unreachable because the function always returns from parser.parse_args() on line 189. This dead code should be removed.

heidi-dang and others added 2 commits February 20, 2026 23:12

test: skip daemon tests when heidid missing

45ea13c

gemini-code-assist bot reviewed Feb 20, 2026

View reviewed changes

chore: retrigger CI

26f7525

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

test: skip daemon tests when heidid missing#79

test: skip daemon tests when heidid missing#79
heidi-dang wants to merge 3 commits intomainfrom
fix/ci-skip-daemon-without-heidid

heidi-dang commented Feb 20, 2026

Uh oh!

gemini-code-assist bot commented Feb 20, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Feb 20, 2026

Uh oh!

gemini-code-assist bot Feb 20, 2026

Uh oh!

gemini-code-assist bot Feb 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

Conversation

heidi-dang commented Feb 20, 2026

Uh oh!

gemini-code-assist bot commented Feb 20, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant