Add custom recency test with optional timestamp support (#220) by hunterpack · Pull Request #224 · stellar/stellar-dbt-public

hunterpack · 2026-04-07T19:32:27Z

Add custom recency test with optional timestamp_val parameter

Replace dbt_utils.recency with a custom recency test that accepts an optional timestamp_val to set the threshold relative to a specific timestamp instead of current_timestamp(). All 51 model recency tests now use this custom test with timestamp_val set to batch_end_date.

lint

PR Checklist

PR Structure

This PR has reasonably narrow scope (if not, break it down into smaller PRs).
This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
This PR's title starts with the jira ticket associated with the PR.

Thoroughness

This PR adds tests for the most critical parts of the new functionality or fixes.
I've updated the docs and README with the added features, breaking changes, new instructions on how to use the repository.

Release planning

I've decided if this PR requires a new major/minor/patch version accordingly to
semver, and I've changed the name of the BRANCH to major/* , minor/* or patch/* .

What

[TODO: Short statement about what is changing.]

Why

[TODO: Why this change is being made. Include any context required to understand the why.]

Known limitations

[TODO or N/A]

* Add custom recency test with optional timestamp_val parameter Replace dbt_utils.recency with a custom recency test that accepts an optional timestamp_val to set the threshold relative to a specific timestamp instead of current_timestamp(). All 51 model recency tests now use this custom test with timestamp_val set to batch_end_date. * lint

Copilot

Pull request overview

This PR replaces dbt_utils.recency usage across the project with a project-local generic recency test that supports an optional timestamp_val parameter, enabling recency to be evaluated relative to var("batch_end_date") rather than current_timestamp() (useful for batched/backfill runs).

Changes:

Added a custom generic recency test (tests/generic/recency.sql) with optional timestamp_val.
Updated model/snapshot YAMLs to use recency and set timestamp_val: '{{ var("batch_end_date") }}'.
Applied lint/formatting adjustments related to these edits.

Reviewed changes

Copilot reviewed 52 out of 52 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/generic/recency.sql	Introduces custom recency generic test with optional timestamp anchoring.
models/staging/stg_ttl.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_trust_lines.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_token_transfers_raw.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_restored_key.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_offers.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_liquidity_pools.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_history_transactions.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_history_trades.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_history_operations.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_history_ledgers.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_history_effects.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_history_contract_events.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_history_assets.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_contract_data.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_claimable_balances.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_accounts.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/staging/stg_account_signers.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/trustlines_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/reflector_prices_data_sdex_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/reflector_prices_data_fex_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/reflector_prices_data_cex_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/liquidity_pools_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/evicted_keys_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/contract_data_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/snapshots/accounts_snapshot.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/tvl/tvl_agg.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/trade_agg.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/tokens/transfers/token_transfers.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_fee_stats_agg.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/ttl_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/trust_lines_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/offers_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/liquidity_pools_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/contract_data_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/contract_code_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/claimable_balances_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/accounts_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/ledger_current_state/account_signers_current.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/hourly_soroban_fee_agg_contract.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/hourly_fee_agg_account.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/history_assets.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/fee_stats_agg.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/evicted_keys.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/enriched_history/enriched_history_operations.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/enriched_history/enriched_history_operations_soroban.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/daily_fee_stats_agg.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/asset_balances/asset_balances__daily_agg.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/marts/account_balances/account_balances__daily_agg.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/intermediate/reflector_prices/int_reflector_prices__sdex.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/intermediate/reflector_prices/int_reflector_prices__fex.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.
models/intermediate/reflector_prices/int_reflector_prices__cex.yml	Switches to custom `recency` and anchors threshold to `batch_end_date`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-07T19:36:42Z

tests/generic/recency.sql

+{% if group_by_columns|length() > 0 %}
+  {% set select_gb_cols = group_by_columns|join(' ,') + ', ' %}
+  {% set groupby_gb_cols = 'group by ' + group_by_columns|join(',') %}
+{% endif %}


select_gb_cols and groupby_gb_cols are only set inside the {% if group_by_columns|length() > 0 %} block, but they’re referenced unconditionally later. With the default group_by_columns=[] (which all current recency test invocations appear to use), this will raise an undefined variable error at compile time. Initialize both variables to empty strings before the if block (or use |default('')) so the test works without grouping.

Copilot · 2026-04-07T19:36:43Z

tests/generic/recency.sql

+  {% set select_gb_cols = group_by_columns|join(' ,') + ', ' %}
+  {% set groupby_gb_cols = 'group by ' + group_by_columns|join(',') %}


group_by_columns|join(' ,') produces SQL like col1 ,col2 (space before the comma). This is unusual formatting and may trigger SQL linting rules; consider using a standard delimiter like ', ' for both the SELECT list and GROUP BY list for consistent formatting.

hunterpack requested a review from a team as a code owner April 7, 2026 19:32

Copilot AI review requested due to automatic review settings April 7, 2026 19:32

Copilot started reviewing on behalf of hunterpack April 7, 2026 19:33 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

amishas157 approved these changes Apr 7, 2026

View reviewed changes

hunterpack merged commit b27f498 into master Apr 7, 2026
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add custom recency test with optional timestamp support (#220)#224

Add custom recency test with optional timestamp support (#220)#224
hunterpack merged 1 commit intomasterfrom
release-v20260407

hunterpack commented Apr 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		{% set select_gb_cols = group_by_columns\|join(' ,') + ', ' %}
		{% set groupby_gb_cols = 'group by ' + group_by_columns\|join(',') %}

Conversation

hunterpack commented Apr 7, 2026

PR Structure

Thoroughness

Release planning

What

Why

Known limitations

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants