-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
Problem
When test files fail to load due to syntax errors, the testing framework silently skips them without any indication of failure. This makes it difficult to detect problems with test file syntax.
Current Behavior
- Syntax errors in test files cause no tests to be discovered
- Framework continues and exits with code 0 (success)
- No clear error message indicating the problem
- User assumes tests ran successfully when they actually failed to load
Example
$ scryer-prolog -f -g run_tests syntax_error_test.pl
error(syntax_error(incomplete_reduction),read_term/3:8).
Running tests in module user.
$ echo $?
0Solution
Detect when no tests are found in any module and report this as an error:
- Check if any tests were discovered after loading
- Display clear red error message: "ERROR: No tests found. This may indicate a file loading error."
- Return exit code 1 (indicating failure)
- Maintain backward compatibility with valid test files
Result
$ scryer-prolog -f -g run_tests syntax_error_test.pl
error(syntax_error(incomplete_reduction),read_term/3:8).
ERROR: No tests found. This may indicate a file loading error.
$ echo $?
1Benefits
- Catches syntax errors in test files immediately
- Clear feedback to developers
- Proper exit codes for CI/CD integration
- No impact on valid test files
Metadata
Metadata
Assignees
Labels
No labels