Skip to content

Commit fff6114

Browse files
pieternpietern
authored
Merge branch 'main' into tf-bump-next
2 parents 357f036 + 6f7b510 commit fff6114

File tree

34 files changed

+947
-1506
lines changed

34 files changed

+947
-1506
lines changed

‎.wsignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ python/docs/images/databricks-logo.svg
1111
**/*.zip
1212
**/*.whl
1313
**/*.png
14-
**/*.tgz
1514

1615
# new lines are recorded differently on windows and unix.
1716
# In unix: "raw_body": "hello, world\n"

‎NEXT_CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@
77
### CLI
88

99
### Bundles
10+
* `bundle deployment migrate` will now run a plan before migration to check if deployment was done ([#4088](https://github.com/databricks/cli/pull/4088))
11+
* engine/direct: Increase "serial" in state file after every deployment ([#4115](https://github.com/databricks/cli/pull/4115))
1012

1113
### Dependency updates
1214

‎acceptance/bundle/help/bundle-deployment-migrate/output.txt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ Usage:
1414
databricks bundle deployment migrate [flags]
1515

1616
Flags:
17-
-h, --help help for migrate
17+
-h, --help help for migrate
18+
--noplancheck Skip running bundle plan before migration.
1819

1920
Global Flags:
2021
--debug enable debug logging

‎acceptance/bundle/migrate/basic/out.new_state.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"lineage": "[UUID]",
3-
"serial": 5,
3+
"serial": 6,
44
"state": {
55
"resources.jobs.test_job": {
66
"__id__": "[NUMID]",

‎acceptance/bundle/migrate/basic/output.txt‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,11 @@ Updating deployment state...
1111
Deployment complete!
1212

1313
>>> [CLI] bundle deployment migrate
14+
Note: Migration should be done after a full deploy. Running plan now to verify that deployment was done:
15+
Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
1416
Migrated 3 resources to direct engine state file: [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json
1517

16-
Validate the migration by running "bundle plan", there should be no actions planned.
18+
Validate the migration by running "databricks bundle plan", there should be no actions planned.
1719

1820
The state file is not synchronized to the workspace yet. To do that and finalize the migration, run "bundle deploy".
1921

@@ -30,21 +32,21 @@ Error: Required engine "terraform" does not match present state files. Clear "DA
3032

3133
Available state files:
3234
- terraform.tfstate: remote terraform state serial=4 lineage="[UUID]"
33-
- [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=5 lineage="[UUID]"
35+
- [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=6 lineage="[UUID]"
3436

3537

3638
>>> DATABRICKS_BUNDLE_ENGINE=terraform musterr [CLI] bundle deploy
3739
Error: Required engine "terraform" does not match present state files. Clear "DATABRICKS_BUNDLE_ENGINE" env var to use engine appropriate for the state.
3840

3941
Available state files:
4042
- terraform.tfstate: remote terraform state serial=4 lineage="[UUID]"
41-
- [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=5 lineage="[UUID]"
43+
- [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=6 lineage="[UUID]"
4244

4345

4446
=== Should show that it's already migrated
4547
>>> musterr [CLI] bundle deployment migrate
4648
Error: already using direct engine
47-
Details: [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=5 lineage="[UUID]"
49+
Details: [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=6 lineage="[UUID]"
4850

4951
>>> DATABRICKS_BUNDLE_ENGINE=direct [CLI] bundle plan
5052
update pipelines.test_pipeline
@@ -95,14 +97,14 @@ Deployment complete!
9597
=== Should show that it's already migrated
9698
>>> musterr [CLI] bundle deployment migrate
9799
Error: already using direct engine
98-
Details: [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=5 lineage="[UUID]"
100+
Details: [TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=7 lineage="[UUID]"
99101

100102
>>> DATABRICKS_BUNDLE_ENGINE= [CLI] bundle debug states
101-
[TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=5 lineage="[UUID]"
103+
[TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=7 lineage="[UUID]"
102104

103105
>>> DATABRICKS_BUNDLE_ENGINE= [CLI] bundle debug states --force-pull
104-
resources.json: remote direct state serial=5 lineage="[UUID]"
105-
[TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=5 lineage="[UUID]"
106+
resources.json: remote direct state serial=7 lineage="[UUID]"
107+
[TEST_TMP_DIR]/.databricks/bundle/dev/resources.json: local direct state serial=7 lineage="[UUID]"
106108

107109
=== Extra plan: should have no drift
108110
>>> DATABRICKS_BUNDLE_ENGINE= [CLI] bundle plan

‎acceptance/bundle/migrate/dashboards/out.new_state.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"lineage": "[UUID]",
3-
"serial": 2,
3+
"serial": 3,
44
"state": {
55
"resources.dashboards.dashboard1": {
66
"__id__": "[DASHBOARD_ID]",

‎acceptance/bundle/migrate/dashboards/output.txt‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,11 @@ Updating deployment state...
66
Deployment complete!
77

88
>>> [CLI] bundle deployment migrate
9+
Note: Migration should be done after a full deploy. Running plan now to verify that deployment was done:
10+
Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
911
Migrated 1 resources to direct engine state file: [TEST_TMP_DIR]/.databricks/bundle/default/resources.json
1012

11-
Validate the migration by running "bundle plan", there should be no actions planned.
13+
Validate the migration by running "databricks bundle plan", there should be no actions planned.
1214

1315
The state file is not synchronized to the workspace yet. To do that and finalize the migration, run "bundle deploy".
1416

@@ -20,15 +22,15 @@ Error: Required engine "terraform" does not match present state files. Clear "DA
2022

2123
Available state files:
2224
- terraform.tfstate: remote terraform state serial=1 lineage="[UUID]"
23-
- [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=2 lineage="[UUID]"
25+
- [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=3 lineage="[UUID]"
2426

2527

2628
>>> DATABRICKS_BUNDLE_ENGINE=terraform musterr [CLI] bundle deploy
2729
Error: Required engine "terraform" does not match present state files. Clear "DATABRICKS_BUNDLE_ENGINE" env var to use engine appropriate for the state.
2830

2931
Available state files:
3032
- terraform.tfstate: remote terraform state serial=1 lineage="[UUID]"
31-
- [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=2 lineage="[UUID]"
33+
- [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=3 lineage="[UUID]"
3234

3335

3436
>>> DATABRICKS_BUNDLE_ENGINE=direct [CLI] bundle plan
@@ -52,11 +54,11 @@ Deployment complete!
5254
=== Should show that it's already migrated
5355
>>> musterr [CLI] bundle deployment migrate
5456
Error: already using direct engine
55-
Details: [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=2 lineage="[UUID]"
57+
Details: [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=4 lineage="[UUID]"
5658

5759
>>> DATABRICKS_BUNDLE_ENGINE= [CLI] bundle debug states
58-
[TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=2 lineage="[UUID]"
60+
[TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=4 lineage="[UUID]"
5961

6062
>>> DATABRICKS_BUNDLE_ENGINE= [CLI] bundle debug states --force-pull
61-
resources.json: remote direct state serial=2 lineage="[UUID]"
62-
[TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=2 lineage="[UUID]"
63+
resources.json: remote direct state serial=4 lineage="[UUID]"
64+
[TEST_TMP_DIR]/.databricks/bundle/default/resources.json: local direct state serial=4 lineage="[UUID]"

‎acceptance/bundle/migrate/default-python/out.state_after_migration.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"lineage": "[UUID]",
3-
"serial": 4,
3+
"serial": 5,
44
"state": {
55
"resources.jobs.sample_job": {
66
"__id__": "[NUMID]",

‎acceptance/bundle/migrate/default-python/output.txt‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,20 @@ Deployment complete!
2222

2323
>>> print_state.py
2424

25-
>>> [CLI] bundle deployment migrate
25+
>>> musterr [CLI] bundle deployment migrate
26+
Building python_artifact...
27+
Note: Migration should be done after a full deploy. Running plan now to verify that deployment was done:
28+
Building python_artifact...
29+
update jobs.sample_job
30+
31+
Plan: 0 to add, 1 to change, 0 to delete, 1 unchanged
32+
Error: 'databricks bundle plan' shows actions planned, aborting migration. Please run 'databricks bundle deploy' first to ensure your bundle is up to date, If actions persist after deploy, skip plan check with --noplancheck option
33+
34+
>>> [CLI] bundle deployment migrate --noplancheck
2635
Building python_artifact...
2736
Migrated 2 resources to direct engine state file: [TEST_TMP_DIR]/my_default_python/.databricks/bundle/dev/resources.json
2837

29-
Validate the migration by running "bundle plan", there should be no actions planned.
38+
Validate the migration by running "databricks bundle plan", there should be no actions planned.
3039

3140
The state file is not synchronized to the workspace yet. To do that and finalize the migration, run "bundle deploy".
3241

@@ -126,6 +135,15 @@ Deployment complete!
126135
>>> print_state.py
127136

128137
>>> diff.py ../out.state_after_migration.json ../out.state_after_deploy.json
138+
--- ../out.state_after_migration.json
139+
+++ ../out.state_after_deploy.json
140+
@@ -1,5 +1,5 @@
141+
{
142+
"lineage": "[UUID]",
143+
- "serial": 5,
144+
+ "serial": 6,
145+
"state": {
146+
"resources.jobs.sample_job": {
129147

130148
=== Extra plan: should have no drift
131149
>>> DATABRICKS_BUNDLE_ENGINE= [CLI] bundle plan

‎acceptance/bundle/migrate/default-python/script‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,10 @@ cd my_default_python
33

44
trace DATABRICKS_BUNDLE_ENGINE=terraform $CLI bundle deploy
55
trace print_state.py > ../out.state_original.json
6-
trace $CLI bundle deployment migrate
6+
7+
trace musterr $CLI bundle deployment migrate
8+
trace $CLI bundle deployment migrate --noplancheck
9+
710
trace print_state.py > ../out.state_after_migration.json
811
trace jq '.. | .libraries? | select(.)' ../out.state_after_migration.json
912

0 commit comments

Comments
 (0)