Background
flagd#1874 documents that custom operators must return null (not false) on error to trigger the default variant fallback in JSONLogic. The flagd-evaluator already implements the correct behavior for all operators, but error-path behavior is not covered by the shared gherkin test suite.
Missing Test Coverage
sem_ver — invalid version string
Rule: {"sem_ver": [{"var": "version"}, "=", "not-a-version"]}
Expected: returns null → default variant fallback.
sem_ver — unknown operator
Rule: {"sem_ver": [{"var": "version"}, "!=", "1.0.0"]} with an unknown comparator.
Expected: returns null → default variant fallback.
fractional — no bucket matched
When weights are misconfigured and no bucket is matched.
Expected: returns null → default variant fallback (not empty string or exception).
fractional — invalid input
Empty bucket list, missing bucket key, etc.
Expected: returns null → default variant fallback.
What Needs to Be Done
- Add flag definitions to the testbed for error-path scenarios
- Add
@operator-errors gherkin scenarios covering each error path
- Add those scenarios to the evaluator's
tests/gherkin_tests.rs
Related
Background
flagd#1874 documents that custom operators must return
null(notfalse) on error to trigger the default variant fallback in JSONLogic. The flagd-evaluator already implements the correct behavior for all operators, but error-path behavior is not covered by the shared gherkin test suite.Missing Test Coverage
sem_ver— invalid version stringRule:
{"sem_ver": [{"var": "version"}, "=", "not-a-version"]}Expected: returns
null→ default variant fallback.sem_ver— unknown operatorRule:
{"sem_ver": [{"var": "version"}, "!=", "1.0.0"]}with an unknown comparator.Expected: returns
null→ default variant fallback.fractional— no bucket matchedWhen weights are misconfigured and no bucket is matched.
Expected: returns
null→ default variant fallback (not empty string or exception).fractional— invalid inputEmpty bucket list, missing bucket key, etc.
Expected: returns
null→ default variant fallback.What Needs to Be Done
@operator-errorsgherkin scenarios covering each error pathtests/gherkin_tests.rsRelated