Merged
Conversation
Comment on lines
-49
to
-59
| +column_types: | ||
| id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
| account_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
| campaign_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
| ad_set_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
| ad_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
| creative_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
| page_link: "{{ 'string' if target.type in ['bigquery','spark','databricks'] else 'varchar' }}" | ||
| template_page_link: "{{ 'string' if target.type in ['bigquery','spark','databricks'] else 'varchar' }}" | ||
| _fivetran_synced: "timestamp" | ||
| updated_time: "timestamp" |
Contributor
Author
There was a problem hiding this comment.
Broke these out to individual tables to clean up the buildkite messages.
4 tasks
| dbt source freshness --target "$db" || echo "...Only verifying freshness runs…" | ||
| dbt run --target "$db" --full-refresh | ||
| dbt test --target "$db" | ||
| if [ "$db" = "bigquery" ] || [ "$db" = "redshift" ] || [ "$db" = "postgres" ] || [ "$db" = "snowflake" ]; then |
Contributor
Author
There was a problem hiding this comment.
this is to test both string and json versions of url_tags.
fivetran-jamie
approved these changes
Feb 26, 2026
Comment on lines
+93
to
+97
| # Load as VARCHAR first, then convert to native JSON type in a post-hook. | ||
| # This is necessary for testing native JSON datatypes with seed data. | ||
| +post-hook: | ||
| - "alter table {{ this }} add column url_tags {{ 'super' if target.type == 'redshift' else 'variant' if target.type == 'snowflake' else 'json' }} {{ 'default null' if target.type != 'bigquery' }}" | ||
| - "update {{ this }} set url_tags = {{ 'json_parse(url_tags_string)' if target.type == 'redshift' else 'url_tags_string::jsonb' if target.type == 'postgres' else 'parse_json(url_tags_string)' }} {{ 'where url_tags_string is not null' if target.type == 'bigquery' }}" |
Contributor
There was a problem hiding this comment.
Ah so just for my reference, does dbt not support configuring +column_types as JSONs?
Contributor
Author
There was a problem hiding this comment.
@fivetran-jamie good question! dbt does support it, but when I seeded directly to a JSON type, the seeds consistently ended up with unexpected characters. Also Snowflake just consistently error with the Variant type. This approach was the most reliable way I found to avoid introducing those characters. I also validated the resulting seeded table against the connector data sourced from JSON to make sure the extra characters shouldn't be there.
4 tasks
Corrects the description of the integrity test fix in CHANGELOG.
Co-authored-by: Savage Fivetran <sarah.savage@fivetran.com>
* Readme and persist docs update * persist_docs conditional --------- Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Updates the integrity test to validate both 'conversions_value' and 'conversions'.
fivetran-catfritz
added
docs:ready
fivetran-catfritz
added
docs:ready
fivetran-catfritz
added
docs:ready
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Overview
Package version introduced in this PR:
This PR addresses the following Issue/Feature(s):
get_url_tag_querymacro for redshift #29Summary of changes:
url_tagscolumns are stored as JSON/JSONB/VARIANT/SUPER instead of strings.Submission Checklist
Changelog