Skip to content

Commit 32295be

Browse files
shreyas-goenkashreyas-goenka
authored
Add support for .dbalert.json files (#3602)
## Changes Adds support for `.dbalert.json` files to define alert configurations separately from `databricks.yml`. **Key additions:** - New `file_path` field for alerts in bundle config - `LoadDBAlertFiles` mutator to read and merge `.dbalert.json` files - Support for `query_lines` and `custom_description_lines` fields in .dbalert.json files. - Validation that only `warehouse_id`, `display_name`, and `file_path` can be set in YAML when using a file - Error handling for variable interpolations in `.dbalert.json` files ## Tests - 3 acceptance tests covering successful file loading, disallowed field errors, and variable interpolation errors - Unit test for `LoadDBAlertFiles` mutator - Tests run with both `terraform` and `direct` deployment engines
1 parent 4312330 commit 32295be

File tree

28 files changed

+640
-1
lines changed

28 files changed

+640
-1
lines changed

‎.golangci.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ linters:
9090
- legacy
9191
- std-error-handling
9292
rules:
93-
- path-except: bundle/direct/dresources
93+
- path-except: bundle/direct/dresources|bundle/config/mutator/load_dbalert_files.go
9494
linters:
9595
- exhaustruct
9696
- path: bundle/direct/dresources/.*_test.go

‎acceptance/bundle/refschema/out.fields.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ resources.alerts.*.evaluation.threshold.value *sql.AlertV2OperandValue ALL
3030
resources.alerts.*.evaluation.threshold.value.bool_value bool ALL
3131
resources.alerts.*.evaluation.threshold.value.double_value float64 ALL
3232
resources.alerts.*.evaluation.threshold.value.string_value string ALL
33+
resources.alerts.*.file_path string INPUT
3334
resources.alerts.*.id string ALL
3435
resources.alerts.*.lifecycle resources.Lifecycle INPUT
3536
resources.alerts.*.lifecycle.prevent_destroy bool INPUT

‎acceptance/bundle/resources/alerts/with_file/alert.dbalert.json‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"custom_summary": "My alert from file",
3+
"evaluation": {
4+
"source": {
5+
"name": "1",
6+
"display": "1",
7+
"aggregation": "MAX"
8+
},
9+
"comparison_operator": "EQUAL",
10+
"threshold": {
11+
"value": {
12+
"double_value": 2.0
13+
}
14+
},
15+
"notification": {
16+
"retrigger_seconds": 1,
17+
"notify_on_ok": false
18+
}
19+
},
20+
"schedule": {
21+
"quartz_cron_schedule": "44 19 */1 * * ?",
22+
"timezone_id": "Europe/Amsterdam"
23+
},
24+
"query_lines": [
25+
"select 2"
26+
],
27+
"custom_description_lines": [
28+
"My alert from file",
29+
"with multiple lines",
30+
"and a third line"
31+
]
32+
}

‎acceptance/bundle/resources/alerts/with_file/databricks.yml.tmpl‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
bundle:
2+
name: alerts-with-file-$UNIQUE_NAME
3+
4+
resources:
5+
alerts:
6+
myalert:
7+
warehouse_id: $TEST_DEFAULT_WAREHOUSE_ID
8+
display_name: "myalert"
9+
file_path: ./alert.dbalert.json

‎acceptance/bundle/resources/alerts/with_file/out.test.toml‎

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎acceptance/bundle/resources/alerts/with_file/output.txt‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
2+
>>> [CLI] bundle validate -o json
3+
{
4+
"custom_description": "My alert from file\nwith multiple lines\nand a third line\n",
5+
"custom_summary": "My alert from file",
6+
"display_name": "myalert",
7+
"evaluation": {
8+
"comparison_operator": "EQUAL",
9+
"notification": {
10+
"notify_on_ok": false,
11+
"retrigger_seconds": 1
12+
},
13+
"source": {
14+
"aggregation": "MAX",
15+
"display": "1",
16+
"name": "1"
17+
},
18+
"threshold": {
19+
"value": {
20+
"double_value": 2
21+
}
22+
}
23+
},
24+
"file_path": "alert.dbalert.json",
25+
"parent_path": "/Workspace/Users/[USERNAME]/.bundle/alerts-with-file-[UNIQUE_NAME]/default/resources",
26+
"query_text": "select 2\n",
27+
"schedule": {
28+
"quartz_cron_schedule": "44 19 */1 * * ?",
29+
"timezone_id": "Europe/Amsterdam"
30+
},
31+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
32+
}
33+
34+
>>> [CLI] bundle deploy
35+
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/alerts-with-file-[UNIQUE_NAME]/default/files...
36+
Deploying resources...
37+
Updating deployment state...
38+
Deployment complete!
39+
40+
>>> [CLI] alerts-v2 get-alert [ALERT_ID]
41+
{
42+
"display_name": "myalert",
43+
"lifecycle_state": "ACTIVE",
44+
"custom_summary": "My alert from file",
45+
"evaluation": {
46+
"comparison_operator": "EQUAL",
47+
"notification": {
48+
"notify_on_ok": false,
49+
"retrigger_seconds": 1
50+
},
51+
"source": {
52+
"aggregation": "MAX",
53+
"display": "1",
54+
"name": "1"
55+
},
56+
"threshold": {
57+
"value": {
58+
"double_value": 2
59+
}
60+
}
61+
},
62+
"query_text": "select 2\n",
63+
"schedule": {
64+
"quartz_cron_schedule": "44 19 */1 * * ?",
65+
"timezone_id": "Europe/Amsterdam"
66+
},
67+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
68+
}
69+
70+
=== assert that the uploaded alert file is the same as the local one
71+
>>> [CLI] workspace export /Workspace/Users/[USERNAME]/.bundle/alerts-with-file-[UNIQUE_NAME]/default/resources/myalert.dbalert.json
72+
73+
>>> diff exported_alert.dbalert.json alert.dbalert.json --strip-trailing-cr
74+
75+
>>> [CLI] alerts-v2 get-alert [ALERT_ID]
76+
{
77+
"display_name": "myalert",
78+
"lifecycle_state": "ACTIVE"
79+
}
80+
81+
>>> [CLI] bundle summary
82+
Name: alerts-with-file-[UNIQUE_NAME]
83+
Target: default
84+
Workspace:
85+
User: [USERNAME]
86+
Path: /Workspace/Users/[USERNAME]/.bundle/alerts-with-file-[UNIQUE_NAME]/default
87+
Resources:
88+
Alerts:
89+
myalert:
90+
Name: myalert
91+
URL: [DATABRICKS_URL]/sql/alerts-v2/[ALERT_ID]
92+
93+
>>> [CLI] bundle destroy --auto-approve
94+
The following resources will be deleted:
95+
delete resources.alerts.myalert
96+
97+
All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/alerts-with-file-[UNIQUE_NAME]/default
98+
99+
Deleting files...
100+
Destroy complete!

‎acceptance/bundle/resources/alerts/with_file/script‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
envsubst < databricks.yml.tmpl > databricks.yml
2+
3+
4+
cleanup() {
5+
trace $CLI bundle destroy --auto-approve
6+
}
7+
trap cleanup EXIT
8+
9+
trace $CLI bundle validate -o json | jq .resources.alerts.myalert
10+
11+
trace $CLI bundle deploy
12+
13+
alert_id=$($CLI bundle summary --output json | jq -r '.resources.alerts.myalert.id')
14+
15+
echo "$alert_id:ALERT_ID" >> ACC_REPLS
16+
17+
trace $CLI alerts-v2 get-alert $alert_id | jq '{display_name, lifecycle_state, custom_summary, evaluation, query_text, schedule, warehouse_id}'
18+
19+
alert_path=$($CLI bundle summary --output json | jq -r '.resources.alerts.myalert.parent_path')/myalert.dbalert.json
20+
title "assert that the uploaded alert file is the same as the local one"
21+
trace $CLI workspace export $alert_path > exported_alert.dbalert.json
22+
trace diff exported_alert.dbalert.json alert.dbalert.json --strip-trailing-cr
23+
rm exported_alert.dbalert.json
24+
25+
trace $CLI alerts-v2 get-alert $alert_id | jq '{display_name, lifecycle_state}'
26+
27+
trace $CLI bundle summary

‎acceptance/bundle/resources/alerts/with_file/test.toml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Local = false
2+
Cloud = true
3+
RecordRequests = false
4+
Ignore = [".databricks"]
5+
6+
# redact ?o=[NUMID]. Different clouds can have different URL serialization, like [DATABRICKS_URL]/sql/alerts-v2/[ALERT_ID] vs [DATABRICKS_URL]/sql/alerts-v2/[ALERT_ID]?o=[NUMID].
7+
[[Repls]]
8+
Old = '\?o=\d+'
9+
New = ''
10+
Order = 0
11+
12+
[EnvMatrix]
13+
DATABRICKS_BUNDLE_ENGINE = ["terraform", "direct"]
14+
15+
[Env]
16+
# MSYS2 automatically converts absolute paths like /Users/$username/$UNIQUE_NAME to
17+
# C:/Program Files/Git/Users/$username/UNIQUE_NAME before passing it to the CLI
18+
# Setting this environment variable prevents that conversion on windows.
19+
MSYS_NO_PATHCONV = "1"

‎acceptance/bundle/resources/alerts/with_file_not_allowed_field_error/alert.dbalert.json‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
{
2+
"custom_summary": "My alert from file",
3+
"evaluation": {
4+
"comparison_operator": "EQUAL",
5+
"notification": {
6+
"notify_on_ok": false,
7+
"retrigger_seconds": 1
8+
},
9+
"source": {
10+
"aggregation": "MAX",
11+
"display": "1",
12+
"name": "1"
13+
},
14+
"threshold": {
15+
"value": {
16+
"double_value": 2
17+
}
18+
}
19+
},
20+
"query_lines": [
21+
"select 2"
22+
],
23+
"custom_description_lines": [
24+
"My alert from file",
25+
"with multiple lines",
26+
"and a third line"
27+
],
28+
"schedule": {
29+
"pause_status": "UNPAUSED",
30+
"quartz_cron_schedule": "44 19 */1 * * ?",
31+
"timezone_id": "Europe/Amsterdam"
32+
}
33+
}

‎acceptance/bundle/resources/alerts/with_file_not_allowed_field_error/databricks.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
bundle:
2+
name: alerts-not-allowed-field
3+
4+
resources:
5+
alerts:
6+
myalert:
7+
warehouse_id: "0123456789012345"
8+
display_name: "My alert"
9+
file_path: ./alert.dbalert.json
10+
# This field is not allowed when file_path is specified
11+
query_text: "select 3"

0 commit comments

Comments
 (0)