Skip to content

tighten multi-file experiment validation#195

Open
CKwin26 wants to merge 1 commit intoaiming-lab:mainfrom
CKwin26:codex/validator-precision
Open

tighten multi-file experiment validation#195
CKwin26 wants to merge 1 commit intoaiming-lab:mainfrom
CKwin26:codex/validator-precision

Conversation

@CKwin26
Copy link
Copy Markdown

@CKwin26 CKwin26 commented Mar 31, 2026

Summary

  • make deep experiment validation catch undefined helper calls such as missing local utility functions
  • avoid flagging legitimate subclasses that inherit __init__ and only implement one core method
  • detect duplicate algorithm-class definitions across multiple generated project files

Why

Generated experiment projects are increasingly multi-file. The validator was missing two important cases:

  • it could miss obvious runtime breakage from undefined helper calls
  • it could misclassify valid inherited subclasses as low-quality classes

At the same time, duplicated class definitions across files like main.py and models.py are a real failure mode for generated projects and should be surfaced explicitly.

Changes

  • add undefined-call checking to deep_validate_files(...)
  • relax the "too few methods" class-quality heuristic for classes with explicit base classes
  • add a dedicated warning when the same non-trivial algorithm/helper class is defined in multiple files
  • avoid emitting the less-useful duplicate-body warning for same-name classes once the cross-file duplicate-class warning is raised

Validation

  • python -m pytest -q tests/test_rc_executor.py -k "deep_validation_detects_undefined_helper_calls or deep_validation_allows_inherited_single_core_method_subclass or deep_validation_detects_duplicate_algorithm_classes_across_files"
  • python3 -m compileall researchclaw/experiment/validator.py tests/test_rc_executor.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant