-
Notifications
You must be signed in to change notification settings - Fork 117
Update name to Declarative Automation Bundles #148
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
9c0934b
e597681
ebde463
7c1c43c
8bb7289
efbaa20
e69fcb7
69184a1
85060be
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| This folder is reserved for Databricks Asset Bundles definitions. | ||
| This folder is reserved for Declarative Automation Bundles definitions. | ||
|
|
||
| New jobs and pipelines should conventions from the 'data-engineering' template. | ||
| See https://github.com/databricks/bundle-examples/blob/main/contrib/templates/data-engineering/README.md. |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # AI Document Processing Job with Structured Streaming | ||||||
|
|
||||||
| A Databricks Asset Bundle demonstrating **incremental document processing** using `ai_parse_document`, `ai_query`, and Databricks Jobs with Structured Streaming. | ||||||
| A Declarative Automation Bundle demonstrating **incremental document processing** using `ai_parse_document`, `ai_query`, and Databricks Jobs with Structured Streaming. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Same here - the product name is plural, so changed the language to work with that. Also a nit to remove the inline-bold AI tell. |
||||||
|
|
||||||
| ## Overview | ||||||
|
|
||||||
|
|
@@ -171,7 +171,7 @@ The included notebook visualizes parsing results with interactive bounding boxes | |||||
|
|
||||||
| ## Resources | ||||||
|
|
||||||
| - [Databricks Asset Bundles](https://docs.databricks.com/dev-tools/bundles/) | ||||||
| - [Declarative Automation Bundles](https://docs.databricks.com/dev-tools/bundles/) | ||||||
| - [Databricks Workflows](https://docs.databricks.com/workflows/) | ||||||
| - [Structured Streaming](https://docs.databricks.com/structured-streaming/) | ||||||
| - [`ai_parse_document` Function](https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_parse_document) | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for ai_parse_document_workflow. | ||||||
| # This is a Declarative Automation Bundle definition for ai_parse_document_workflow. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Same here. |
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: ai_parse_document_workflow | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| This folder is reserved for Databricks Asset Bundles definitions. | ||
| This folder is reserved for Declarative Automation Bundles definitions. | ||
|
|
||
| New jobs and pipelines should conventions from the 'data-engineering' template. | ||
| See https://github.com/databricks/bundle-examples/blob/main/contrib/templates/data-engineering/README.md. |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for {{.project_name}}. | ||||||
| # This is a Declarative Automation Bundle definition for {{.project_name}}. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Same here |
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: {{.project_name}} | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for {{.project_name}}. | ||||||
| # This is a Declarative Automation Bundle definition for {{.project_name}}. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: {{.project_name}} | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
|
|
||
| This folder is reserved for Databricks Asset Bundles resource definitions. | ||
| This folder is reserved for Declarative Automation Bundles resource definitions. | ||
|
|
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for {{.project_name}}. | ||||||
| # This is a Declarative Automation Bundle definition for {{.project_name}}. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: {{.project_name}} | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,7 +1,7 @@ | ||||||
| # Databricks app with OLTP database | ||||||
|
|
||||||
| This example demonstrates how to define a Databricks app backed by | ||||||
| an OLTP Postgres in a Databricks Asset Bundle. | ||||||
| an OLTP Postgres in a Declarative Automation Bundle. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| It includes and deploys an example application that uses Python and Dash and a database instance. | ||||||
| When application is started it provisions its own schema and demonstration data in the OLTP database. | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for app_with_database. | ||||||
| # This is a Declarative Automation Bundle definition for app_with_database. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| bundle: | ||||||
| name: app_with_database_example | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # Dashboard for NYC Taxi Trip Analysis | ||||||
|
|
||||||
| This example shows how to define a Databricks Asset Bundle with an AI/BI dashboard and a job that captures a snapshot of the dashboard and emails it to a subscriber. | ||||||
| This example shows how to define a Declarative Automation Bundle with an AI/BI dashboard and a job that captures a snapshot of the dashboard and emails it to a subscriber. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| It deploys the sample __NYC Taxi Trip Analysis__ dashboard to a Databricks workspace and configures a daily schedule to run the dashboard and send the snapshot in email to a specified email address. | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # OLTP database instance with a catalog | ||||||
|
|
||||||
| This example demonstrates how to define an OLTP database instance and a database catalog in a Databricks Asset Bundle. | ||||||
| This example demonstrates how to define an OLTP database instance and a database catalog in a Declarative Automation Bundle. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| It includes and deploys an example database instance and a catalog. When data changes in the database instance, they are reflected in Unity Catalog. | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # Databricks App for working with Databricks jobs | ||||||
|
|
||||||
| This example demonstrates how to define an Databricks App in a Databricks Asset Bundle. | ||||||
| This example demonstrates how to define an Databricks App in a Declarative Automation Bundle. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| It includes and deploys an example app and a job managed by DABs to a Databricks workspace. | ||||||
| The app shows current status of the job and lists all existing runs. | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # Development cluster | ||||||
|
|
||||||
| This example demonstrates how to define and use a development (all-purpose) cluster in a Databricks Asset Bundle. | ||||||
| This example demonstrates how to define and use a development (all-purpose) cluster in a Declarative Automation Bundle. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| This bundle defines an `example_job` which is run on a job cluster in production mode. | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # job_backfill_data | ||||||
|
|
||||||
| This example demonstrates a Databricks Asset Bundle (DABs) Job that runs a SQL task with a date parameter for backfilling data. | ||||||
| This example demonstrates a Declarative Automation Bundle Job that runs a SQL task with a date parameter for backfilling data. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| The Job consists of: | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for job backfill data. | ||||||
| # This is a Declarative Automation Bundle definition for job backfill data. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: job_backfill_data | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for job conditional execution. | ||||||
| # This is a Declarative Automation Bundle definition for job conditional execution. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Or "...a job that runs based on a condition."? |
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: job_conditional_execution | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for job file arrival. | ||||||
| # This is a Declarative Automation Bundle definition for job file arrival. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: job_file_arrival | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for job programmatic generation. | ||||||
| # This is a Declarative Automation Bundle definition for job programmatic generation. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: job_programmatic_generation | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # Databricks job that reads a secret from a secret scope | ||||||
|
|
||||||
| This example demonstrates how to define a secret scope and a job with a task that reads from it in a Databricks Asset Bundle. | ||||||
| This example demonstrates how to define a secret scope and a job with a task that reads from it in a Declarative Automation Bundle. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| It includes and deploys an example secret scope, and a job with a task in a bundle that reads a secret from the secret scope to a Databricks workspace. | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for job table update trigger. | ||||||
| # This is a Declarative Automation Bundle definition for job table update trigger. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: job_table_update_trigger | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for job with for each. | ||||||
| # This is a Declarative Automation Bundle definition for job with for each. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: job_with_for_each | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # Job with multiple wheels | ||||||
|
|
||||||
| This example demonstrates how to define and use a job with multiple wheel dependencies in a Databricks Asset Bundle. | ||||||
| This example demonstrates how to define and use a job with multiple wheel dependencies in a Declarative Automation Bundle. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| One of the wheel files depends on the other. It is important to specify the order of the wheels in the job such that | ||||||
| the dependent wheel is installed first, since it won't be available in a public registry. | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,4 @@ | ||||||
| # This is a Databricks asset bundle definition for job with task values. | ||||||
| # This is a Declarative Automation Bundle definition for job with task values. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| # See https://docs.databricks.com/dev-tools/bundles/index.html for documentation. | ||||||
| bundle: | ||||||
| name: job_with_task_values | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||
| # Private wheel packages | ||||||
|
|
||||||
| This example demonstrates how to use a private wheel package from a job in a Databricks Asset Bundle. | ||||||
| This Declarative Automation Bundles example demonstrates how to use a private wheel package from a job. | ||||||
|
|
||||||
| If you are using notebooks, you can use the approach documented in [Notebook-scoped Python libraries][doc] to install | ||||||
| wheels from a private repository in a notebook. You can use the workaround documented here if you are not using notebooks. | ||||||
|
|
@@ -15,7 +15,7 @@ wheels from a private repository in a notebook. You can use the workaround docum | |||||
| # Usage | ||||||
|
|
||||||
| You can refer to private wheel files from job libraries or serverless environments by downloading the wheel | ||||||
| and making it part of your Databricks Asset Bundle deployment. | ||||||
| and making it part of your Declarative Automation Bundle deployment. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| To emulate this for this example, we will download a wheel from PyPI, include it in deployment, and refer to it from job configuration. | ||||||
|
|
||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We never do this. The product name is "Declarative Automation Bundles". So either use that (i think it's fine here) or if it's singular, then just use "bundle".