Skip to content

Refactor: Extract Finalsite exclude IDs to intermediate model#3425

Open
anthonygwalters wants to merge 2 commits intomainfrom
claude/add-google-sheets-int-model-Q3ODq
Open

Refactor: Extract Finalsite exclude IDs to intermediate model#3425
anthonygwalters wants to merge 2 commits intomainfrom
claude/add-google-sheets-int-model-Q3ODq

Conversation

@anthonygwalters
Copy link
Copy Markdown
Member

Pull Request

Summary & Motivation

When merged, this pull request will refactor the Finalsite exclude IDs logic into a dedicated intermediate dbt model that combines data from both the Google Sheets staging table and Google Forms responses. This consolidates the exclude IDs logic in one place and makes it reusable across multiple models.

Changes:

  • Created new intermediate model int_google_sheets__finalsite__exclude_ids that unions Finalsite student IDs from the staging table with IDs extracted from a specific Google Form
  • Updated stg_finalsite__status_report to reference the new intermediate model instead of the staging table directly
  • Added corresponding YAML properties file with data tests

Self-review

dbt

  • Include a corresponding [model name].yml properties file for all models
  • Added unique constraint test on finalsite_student_id column with store_failures: true

SQL

  • Used union distinct with appropriate logic for combining two data sources
  • No group by without aggregations
  • No order by in select statements

Test Plan

Existing dbt tests will validate:

  • Uniqueness of finalsite_student_id values in the new intermediate model
  • Referential integrity in stg_finalsite__status_report when filtering by the new model

https://claude.ai/code/session_01E7WESstQxAcGY2ry66oPfN

Union stg_google_sheets__finalsite__exclude_ids with Google Form responses
(form_id = '1iH4Cjy_RIc8TWJf6Ts0sxhpVmCnx4paiT5RSNOS2Rn4') using
UNION DISTINCT to deduplicate exclude IDs from both sources. Update
stg_finalsite__status_report to reference the new intermediate model.

https://claude.ai/code/session_01E7WESstQxAcGY2ry66oPfN
@anthonygwalters anthonygwalters requested a review from a team March 12, 2026 21:07
@cbini
Copy link
Copy Markdown
Member

cbini commented Mar 13, 2026

@anthonygwalters some context? are we migrating the sheet to a form?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants