Skip to content

Commit d17d0ed

Browse files
aepfliCopilot
aepfli
and
Copilot
committed
test(operators): add follow-up error scenarios for operator null-return behaviour
Covers the remaining cases from open-feature/flagd#1874 identified in the review of #342: - starts_with / ends_with: non-string first argument → must return null - starts_with / ends_with: wrong argument count → must return null - sem_ver: wrong argument count (missing target version) → must return null - fractional: all-zero bucket weights (no bucket matched) → must return null - fractional: negative bucket weight clamped to zero → "one" gets effective weight 0, "two" gets 100% of the weight All scenarios use a bare operator as the targeting expression (not wrapped in 'if') so a null result selects the defaultVariant directly. Scenarios are tagged @operator-errors and mirrored in both the SDK-level gherkin (gherkin/targeting.feature) and the evaluator-level gherkin files (evaluator/gherkin/{string,semver,fractional}.feature) with matching flag definitions added to both flags/edge-case-flags.json and evaluator/flags/testkit-flags.json. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Simon Schrottner <simon.schrottner@dynatrace.com>
1 parent fdfac67 commit d17d0ed

6 files changed

Lines changed: 230 additions & 0 deletions

File tree

‎evaluator/flags/testkit-flags.json‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -386,6 +386,70 @@
386386
null
387387
]
388388
}
389+
},
390+
"starts-with-non-string-flag": {
391+
"state": "ENABLED",
392+
"variants": { "fallback": "fallback" },
393+
"defaultVariant": "fallback",
394+
"targeting": {
395+
"starts_with": [{"var": "num"}, "abc"]
396+
}
397+
},
398+
"ends-with-non-string-flag": {
399+
"state": "ENABLED",
400+
"variants": { "fallback": "fallback" },
401+
"defaultVariant": "fallback",
402+
"targeting": {
403+
"ends_with": [{"var": "num"}, "xyz"]
404+
}
405+
},
406+
"starts-with-wrong-args-flag": {
407+
"state": "ENABLED",
408+
"variants": { "fallback": "fallback" },
409+
"defaultVariant": "fallback",
410+
"targeting": {
411+
"starts_with": ["abc"]
412+
}
413+
},
414+
"ends-with-wrong-args-flag": {
415+
"state": "ENABLED",
416+
"variants": { "fallback": "fallback" },
417+
"defaultVariant": "fallback",
418+
"targeting": {
419+
"ends_with": ["xyz"]
420+
}
421+
},
422+
"fractional-zero-weights-flag": {
423+
"state": "ENABLED",
424+
"variants": { "one": "one", "two": "two", "fallback": "fallback" },
425+
"defaultVariant": "fallback",
426+
"targeting": {
427+
"fractional": [
428+
{"var": "targetingKey"},
429+
["one", 0],
430+
["two", 0]
431+
]
432+
}
433+
},
434+
"fractional-negative-weight-flag": {
435+
"state": "ENABLED",
436+
"variants": { "one": "one", "two": "two", "fallback": "fallback" },
437+
"defaultVariant": "fallback",
438+
"targeting": {
439+
"fractional": [
440+
{"var": "targetingKey"},
441+
["one", -50],
442+
["two", 100]
443+
]
444+
}
445+
},
446+
"semver-wrong-args-flag": {
447+
"state": "ENABLED",
448+
"variants": { "fallback": "fallback" },
449+
"defaultVariant": "fallback",
450+
"targeting": {
451+
"sem_ver": [{"var": "version"}, "="]
452+
}
389453
}
390454
},
391455
"$evaluators": {

‎evaluator/gherkin/fractional.feature‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,3 +110,22 @@ Feature: Evaluator fractional operator
110110
| SI7p- | lower |
111111
| 6LvT0 | upper |
112112
| ceQdGm | upper |
113+
114+
# Follow-up error scenarios from https://github.com/open-feature/flagd/issues/1874
115+
116+
@operator-errors
117+
Scenario: fractional with all-zero bucket weights falls back to default variant
118+
Given an evaluator
119+
And a String-flag with key "fractional-zero-weights-flag" and a fallback value "wrong"
120+
And a context containing a targeting key with value "any-user"
121+
When the flag was evaluated with details
122+
Then the resolved details value should be "fallback"
123+
124+
@operator-errors
125+
Scenario: fractional negative bucket weight is clamped to zero
126+
# ["one", -50] is treated as ["one", 0]; "two" gets 100% of the weight
127+
Given an evaluator
128+
And a String-flag with key "fractional-negative-weight-flag" and a fallback value "wrong"
129+
And a context containing a targeting key with value "any-user"
130+
When the flag was evaluated with details
131+
Then the resolved details value should be "two"

‎evaluator/gherkin/semver.feature‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,13 @@ Feature: Evaluator semantic version operator
3131
| 3.0.1 | minor |
3232
| 3.1.0 | major |
3333
| 4.0.0 | none |
34+
35+
# Follow-up error scenarios from https://github.com/open-feature/flagd/issues/1874
36+
37+
@operator-errors
38+
Scenario: sem_ver returns null for wrong argument count
39+
Given an evaluator
40+
And a String-flag with key "semver-wrong-args-flag" and a fallback value "wrong"
41+
And a context containing a key "version", with type "String" and with value "1.0.0"
42+
When the flag was evaluated with details
43+
Then the resolved details value should be "fallback"

‎evaluator/gherkin/string.feature‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,30 @@ Feature: Evaluator string comparison operator
1616
| uvwxyz | postfix |
1717
| abcxyz | prefix |
1818
| lmnopq | none |
19+
20+
# Follow-up error scenarios from https://github.com/open-feature/flagd/issues/1874
21+
# starts_with and ends_with must return null (not false) on error so that the
22+
# default variant is selected, rather than looking up a non-existent "false" variant.
23+
24+
@operator-errors
25+
Scenario Outline: starts_with and ends_with return null for non-string input
26+
Given an evaluator
27+
And a String-flag with key "<key>" and a fallback value "wrong"
28+
And a context containing a key "num", with type "Integer" and with value "123"
29+
When the flag was evaluated with details
30+
Then the resolved details value should be "fallback"
31+
Examples:
32+
| key |
33+
| starts-with-non-string-flag |
34+
| ends-with-non-string-flag |
35+
36+
@operator-errors
37+
Scenario Outline: starts_with and ends_with return null for wrong argument count
38+
Given an evaluator
39+
And a String-flag with key "<key>" and a fallback value "wrong"
40+
When the flag was evaluated with details
41+
Then the resolved details value should be "fallback"
42+
Examples:
43+
| key |
44+
| starts-with-wrong-args-flag |
45+
| ends-with-wrong-args-flag |

‎flags/edge-case-flags.json‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,70 @@
5252
},
5353
"defaultVariant": "false",
5454
"targeting": {}
55+
},
56+
"starts-with-non-string-flag": {
57+
"state": "ENABLED",
58+
"variants": { "fallback": "fallback" },
59+
"defaultVariant": "fallback",
60+
"targeting": {
61+
"starts_with": [{"var": "num"}, "abc"]
62+
}
63+
},
64+
"ends-with-non-string-flag": {
65+
"state": "ENABLED",
66+
"variants": { "fallback": "fallback" },
67+
"defaultVariant": "fallback",
68+
"targeting": {
69+
"ends_with": [{"var": "num"}, "xyz"]
70+
}
71+
},
72+
"starts-with-wrong-args-flag": {
73+
"state": "ENABLED",
74+
"variants": { "fallback": "fallback" },
75+
"defaultVariant": "fallback",
76+
"targeting": {
77+
"starts_with": ["abc"]
78+
}
79+
},
80+
"ends-with-wrong-args-flag": {
81+
"state": "ENABLED",
82+
"variants": { "fallback": "fallback" },
83+
"defaultVariant": "fallback",
84+
"targeting": {
85+
"ends_with": ["xyz"]
86+
}
87+
},
88+
"fractional-zero-weights-flag": {
89+
"state": "ENABLED",
90+
"variants": { "one": "one", "two": "two", "fallback": "fallback" },
91+
"defaultVariant": "fallback",
92+
"targeting": {
93+
"fractional": [
94+
{"var": "targetingKey"},
95+
["one", 0],
96+
["two", 0]
97+
]
98+
}
99+
},
100+
"fractional-negative-weight-flag": {
101+
"state": "ENABLED",
102+
"variants": { "one": "one", "two": "two", "fallback": "fallback" },
103+
"defaultVariant": "fallback",
104+
"targeting": {
105+
"fractional": [
106+
{"var": "targetingKey"},
107+
["one", -50],
108+
["two", 100]
109+
]
110+
}
111+
},
112+
"semver-wrong-args-flag": {
113+
"state": "ENABLED",
114+
"variants": { "fallback": "fallback" },
115+
"defaultVariant": "fallback",
116+
"targeting": {
117+
"sem_ver": [{"var": "version"}, "="]
118+
}
55119
}
56120
}
57121
}

‎gherkin/targeting.feature‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,3 +266,49 @@ Feature: Targeting rules
266266
| missing-variant-targeting-flag | 3 | GENERAL |
267267
| non-string-variant-targeting-flag | 2 | |
268268
| empty-targeting-flag | 1 | |
269+
270+
# Follow-up error scenarios from https://github.com/open-feature/flagd/issues/1874
271+
# Operators must return null (not false) on error so the default variant is selected.
272+
273+
@operator-errors @string
274+
Scenario Outline: starts_with and ends_with return null for non-string input
275+
Given a String-flag with key "<key>" and a default value "wrong"
276+
And a context containing a key "num", with type "Integer" and with value "123"
277+
When the flag was evaluated with details
278+
Then the resolved details value should be "fallback"
279+
Examples:
280+
| key |
281+
| starts-with-non-string-flag |
282+
| ends-with-non-string-flag |
283+
284+
@operator-errors @string
285+
Scenario Outline: starts_with and ends_with return null for wrong argument count
286+
Given a String-flag with key "<key>" and a default value "wrong"
287+
When the flag was evaluated with details
288+
Then the resolved details value should be "fallback"
289+
Examples:
290+
| key |
291+
| starts-with-wrong-args-flag |
292+
| ends-with-wrong-args-flag |
293+
294+
@operator-errors @semver
295+
Scenario: sem_ver returns null for wrong argument count
296+
Given a String-flag with key "semver-wrong-args-flag" and a default value "wrong"
297+
And a context containing a key "version", with type "String" and with value "1.0.0"
298+
When the flag was evaluated with details
299+
Then the resolved details value should be "fallback"
300+
301+
@operator-errors @fractional
302+
Scenario: fractional with all-zero bucket weights falls back to default variant
303+
Given a String-flag with key "fractional-zero-weights-flag" and a default value "wrong"
304+
And a context containing a targeting key with value "any-user"
305+
When the flag was evaluated with details
306+
Then the resolved details value should be "fallback"
307+
308+
@operator-errors @fractional
309+
Scenario: fractional negative bucket weight is clamped to zero
310+
# ["one", -50] is treated as ["one", 0]; "two" gets 100% of the weight
311+
Given a String-flag with key "fractional-negative-weight-flag" and a default value "wrong"
312+
And a context containing a targeting key with value "any-user"
313+
When the flag was evaluated with details
314+
Then the resolved details value should be "two"

0 commit comments

Comments
 (0)