From fb6728fdb312aa3b8ff5b858622819f8102ec5c6 Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Fri, 22 Aug 2025 12:56:55 +0530 Subject: [PATCH 1/8] Create Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 149 ++++++++++++++++++++ 1 file changed, 149 insertions(+) create mode 100644 quickstarts/manage-data/Pushdown-feature.md diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md new file mode 100644 index 00000000000..feee7c4f50b --- /dev/null +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -0,0 +1,149 @@ +# Pushdown Functionality +The Teradata connector supports pushdown for different operations. + +* Join pushdown + + - Cost-based join pushdown + +* Limit pushdown + +* Top N pushdown + +* Projection pushdown + +* Predicate pushdown + +* Aggregate pushdown + +## Join pushdown +Join pushdown allows the Teradata connector to delegate the table join operation to the Teradata. This can result in performance gains and allows Trino to perform the remaining query processing on a smaller amount of data. + +## Cost-based join pushdown +The Teradata connector supports cost-based Join pushdown to make intelligent decisions about whether to push down a join operation to the Teradata. This is based on a catalog configuration property. + +When cost-based join pushdown is enabled, the connector only pushes down join operations if statistics of joining tables are available incase of join pushdown strategy is ```AUTOMATIC```. + +The following table describes catalog configuration properties for join pushdown. + +| Property name | Description | Default value | +|---------------|-------------|---------------| +| join-pushdown.enabled | Enable join pushdown. `join_pushdown_enabled` is the equal catalog session property. | true | +| join-pushdown.strategy | Strategy used to evaluate whether join operations are pushed down. Set to AUTOMATIC to enable cost-based join pushdown, or EAGER to pushdown joins. Note that EAGER can push down joins even when table statistics are unavailable. | AUTOMATIC | + +The connector does not support pushdown of range predicates, such as ```>```, ```<```, or ```BETWEEN```, on columns with character string types like ```CHAR``` or ```VARCHAR```. Equality predicates, such as ```IN``` or ```=```, and inequality predicates, such as ```!=``` on columns with textual types are pushed down. This ensures correctness of results since the remote data source may sort strings differently than Trino. + +## Limit pushdown +A LIMIT reduces the number of returned rows for a SQL SELECT statement. Limit pushdown enables connector to push processing of such queries of unsorted records to Teradata. A pushdown of this clause can improve the performance of the query and significantly reduce the amount of data transferred from Teradata to Trino. + +Teradata converts Trino LIMIT to Teradata equivalent TOP clause. + +```SELECT id, name FROM teradata.public.company LIMIT 5;``` + +Trino query will be converted to + +```SELECT TOP 5 id, name FROM teradata.public.company;``` + +## TOP N pushdown +The combination of a LIMIT with ORDER BY clause creates a small set of records to return out of a large sorted dataset. The pushdown for a query involving LIMIT with ORDER BY is called a Top N pushdown. It enabled the Teradata connector to push the processing of Queries involved in LIMIT with ORDER BY clause to Teradata database, and therefore significantly reduces the amount of data transferred to Trino. + +```SELECT id, name FROM teradata.public.company ORDER BY id LIMIT 5;``` + +Trino query will be converted to Teradata supported syntax + +```SELECT TOP 5 id, name from (SELECT id, name FROM teradata.public.company) as t ORDER BY id;``` + +## Projection pushdown +Projection pushdown optimizes column-based filtering. It uses the column specified in the SELECT clause and other parts of the query to limit access to these columns. The processing is pushed down to the Teradata database by Teradata connector and then Teradata reads and returns only necessary columns. + +Teradata connector inherits the TRINO JDBC framework implementation. Teradata connector doesn’t have any custom implementation with respect to Projection Pushdown. + +## Predicate pushdown +Predicate pushdown optimizes row-based filtering. It uses filtering specified in WHERE clause to omit unnecessary rows. The processing is pushed down to Teradata database by the Teradata connector and Teradata only reads and returns the filter matched rows only. + +The predicate pushdown supports SQL statements with following arithmetic, boolean, comparision operators and functions. + +* = + +* <> + +* < + +* <= + +* \> + +* \>= + +* \+ + +* \- + +* \* + +* / + +* MOD + +* -Value + +* IS NOT NULL + +* NOT + +* IS NULL + +* NULLIF + +* IN + +* LIKE + +* LIKE ESCAPE + +## Aggregate pushdown +Teradata supports Aggregation pushdown for below specified cases. + +### Numeric Data +* CountAll + +* Count + +* Count Distinct + +* Min + +* Max + +* Sum + +* Avg Float + +* Avg Decimal + +* Avg Bigint + +* Statistical Aggregation + + - StddevSamp + + - StddevPop + + - VarianceSamp + + - VariancePop + +* Correlation and Regression Aggregation + + - CovarianceSamp + + - CovariancePop + + - Corr + + - RegrIntercept + + - RegrSlope + +* String Data + +* No Aggregation Pushdown for CHAR and VARCHAR type data From 256ced05201dc8946aace5a4b9f981f74c8b23c6 Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Wed, 10 Sep 2025 13:21:20 +0530 Subject: [PATCH 2/8] Update Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md index feee7c4f50b..82dd13ecbdc 100644 --- a/quickstarts/manage-data/Pushdown-feature.md +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -25,12 +25,12 @@ When cost-based join pushdown is enabled, the connector only pushes down join op The following table describes catalog configuration properties for join pushdown. -| Property name | Description | Default value | -|---------------|-------------|---------------| -| join-pushdown.enabled | Enable join pushdown. `join_pushdown_enabled` is the equal catalog session property. | true | -| join-pushdown.strategy | Strategy used to evaluate whether join operations are pushed down. Set to AUTOMATIC to enable cost-based join pushdown, or EAGER to pushdown joins. Note that EAGER can push down joins even when table statistics are unavailable. | AUTOMATIC | +| Property name | Description | Default value | +|------------------------|-----------------------------------------------------------------------------|---------------| +| join-pushdown.enabled | Enable join pushdown. `join_pushdown_enabled` is the equivalent catalog session property. | true | +| join-pushdown.strategy | Strategy used to evaluate whether join operations are pushed down. Set to `AUTOMATIC` to enable cost-based join pushdown, or `EAGER` to pushdown joins. Note that `EAGER` can push down joins even when table statistics are unavailable. | AUTOMATIC | -The connector does not support pushdown of range predicates, such as ```>```, ```<```, or ```BETWEEN```, on columns with character string types like ```CHAR``` or ```VARCHAR```. Equality predicates, such as ```IN``` or ```=```, and inequality predicates, such as ```!=``` on columns with textual types are pushed down. This ensures correctness of results since the remote data source may sort strings differently than Trino. +The connector does not support pushdown of range predicates, such as ```>```, ```<```, or ```BETWEEN```, on columns with character string types like ```CHAR``` or ```VARCHAR```. Equality predicates, such as ```IN``` or ```=```, and inequality predicates, such as ```!=``` on columns with textual types are pushed down. This ensures correctness of results since the remote data source may sort strings differently than Trino. ## Limit pushdown A LIMIT reduces the number of returned rows for a SQL SELECT statement. Limit pushdown enables connector to push processing of such queries of unsorted records to Teradata. A pushdown of this clause can improve the performance of the query and significantly reduce the amount of data transferred from Teradata to Trino. From 63c4f546abdf3cff2142df9c199babd4dc2a3968 Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Wed, 10 Sep 2025 15:21:54 +0530 Subject: [PATCH 3/8] Update Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 1 + 1 file changed, 1 insertion(+) diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md index 82dd13ecbdc..1ab9de7bbd0 100644 --- a/quickstarts/manage-data/Pushdown-feature.md +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -32,6 +32,7 @@ The following table describes catalog configuration properties for join pushdown The connector does not support pushdown of range predicates, such as ```>```, ```<```, or ```BETWEEN```, on columns with character string types like ```CHAR``` or ```VARCHAR```. Equality predicates, such as ```IN``` or ```=```, and inequality predicates, such as ```!=``` on columns with textual types are pushed down. This ensures correctness of results since the remote data source may sort strings differently than Trino. + ## Limit pushdown A LIMIT reduces the number of returned rows for a SQL SELECT statement. Limit pushdown enables connector to push processing of such queries of unsorted records to Teradata. A pushdown of this clause can improve the performance of the query and significantly reduce the amount of data transferred from Teradata to Trino. From 9cca634e2ee952a021f35a36005203f232b3ca62 Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Wed, 10 Sep 2025 15:53:17 +0530 Subject: [PATCH 4/8] Update Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md index 1ab9de7bbd0..1b7c3bb7993 100644 --- a/quickstarts/manage-data/Pushdown-feature.md +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -25,10 +25,11 @@ When cost-based join pushdown is enabled, the connector only pushes down join op The following table describes catalog configuration properties for join pushdown. -| Property name | Description | Default value | -|------------------------|-----------------------------------------------------------------------------|---------------| -| join-pushdown.enabled | Enable join pushdown. `join_pushdown_enabled` is the equivalent catalog session property. | true | -| join-pushdown.strategy | Strategy used to evaluate whether join operations are pushed down. Set to `AUTOMATIC` to enable cost-based join pushdown, or `EAGER` to pushdown joins. Note that `EAGER` can push down joins even when table statistics are unavailable. | AUTOMATIC | +| Header 1 | Header 2 | Header 3 | +|----------|----------|----------| +| Row 1 | Row 1 | Row 1 | +| Row 2 | Row 2 | Row 2 | +| Row 3 | Row 3 | Row 3 | The connector does not support pushdown of range predicates, such as ```>```, ```<```, or ```BETWEEN```, on columns with character string types like ```CHAR``` or ```VARCHAR```. Equality predicates, such as ```IN``` or ```=```, and inequality predicates, such as ```!=``` on columns with textual types are pushed down. This ensures correctness of results since the remote data source may sort strings differently than Trino. From 129d02d502882d54292448e49eaed761d55b8f31 Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Wed, 10 Sep 2025 15:58:18 +0530 Subject: [PATCH 5/8] Update Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md index 1b7c3bb7993..7f334b38df2 100644 --- a/quickstarts/manage-data/Pushdown-feature.md +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -25,11 +25,10 @@ When cost-based join pushdown is enabled, the connector only pushes down join op The following table describes catalog configuration properties for join pushdown. -| Header 1 | Header 2 | Header 3 | -|----------|----------|----------| -| Row 1 | Row 1 | Row 1 | -| Row 2 | Row 2 | Row 2 | -| Row 3 | Row 3 | Row 3 | +| Property name | Description | Default value | +|-------------------------|-----------------------------------------------------------------------------|---------------| +| join-pushdown.enabled | Enable join pushdown. `join_pushdown_enabled` is the equivalent catalog session property. | true | +| join-pushdown.strategy | Strategy used to evaluate whether join operations are pushed down. Set to `AUTOMATIC` to enable cost-based join pushdown, or `EAGER` to pushdown joins. Note that `EAGER` can push down joins even when table statistics are unavailable. | AUTOMATIC | The connector does not support pushdown of range predicates, such as ```>```, ```<```, or ```BETWEEN```, on columns with character string types like ```CHAR``` or ```VARCHAR```. Equality predicates, such as ```IN``` or ```=```, and inequality predicates, such as ```!=``` on columns with textual types are pushed down. This ensures correctness of results since the remote data source may sort strings differently than Trino. @@ -64,7 +63,7 @@ Predicate pushdown optimizes row-based filtering. It uses filtering specified in The predicate pushdown supports SQL statements with following arithmetic, boolean, comparision operators and functions. -* = +* \= * <> From 7bb4a3ec19f65cd6c9f6a098f69fde16075368a0 Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Wed, 10 Sep 2025 16:02:32 +0530 Subject: [PATCH 6/8] Update Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md index 7f334b38df2..a1e26cb9f07 100644 --- a/quickstarts/manage-data/Pushdown-feature.md +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -69,7 +69,7 @@ The predicate pushdown supports SQL statements with following arithmetic, boolea * < -* <= +* <\= * \> From 6b3b19ff1e4277dcef9c7d95596f6af66254ed33 Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Wed, 10 Sep 2025 16:08:00 +0530 Subject: [PATCH 7/8] Update Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md index a1e26cb9f07..a87f64bbb6a 100644 --- a/quickstarts/manage-data/Pushdown-feature.md +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -63,13 +63,13 @@ Predicate pushdown optimizes row-based filtering. It uses filtering specified in The predicate pushdown supports SQL statements with following arithmetic, boolean, comparision operators and functions. -* \= +* = * <> * < -* <\= +* <= * \> From c8d5304a8d9d25f6641f8f374a390a9f5e15e24b Mon Sep 17 00:00:00 2001 From: Vasudha Dabbiru Date: Wed, 10 Sep 2025 16:16:17 +0530 Subject: [PATCH 8/8] Update Pushdown-feature.md --- quickstarts/manage-data/Pushdown-feature.md | 38 ++++++++++----------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/quickstarts/manage-data/Pushdown-feature.md b/quickstarts/manage-data/Pushdown-feature.md index a87f64bbb6a..dc3781cf037 100644 --- a/quickstarts/manage-data/Pushdown-feature.md +++ b/quickstarts/manage-data/Pushdown-feature.md @@ -63,43 +63,43 @@ Predicate pushdown optimizes row-based filtering. It uses filtering specified in The predicate pushdown supports SQL statements with following arithmetic, boolean, comparision operators and functions. -* = +* `=` -* <> +* `<>` -* < +* `<` -* <= +* `<=` -* \> +* `>` -* \>= +* `>=` -* \+ +* `+` -* \- +* `-` -* \* +* `*` -* / +* `/` -* MOD +* `MOD` -* -Value +* `-Value` -* IS NOT NULL +* `IS NOT NULL` -* NOT +* `NOT` -* IS NULL +* `IS NULL` -* NULLIF +* `NULLIF` -* IN +* `IN` -* LIKE +* `LIKE` -* LIKE ESCAPE +* `LIKE ESCAPE` ## Aggregate pushdown Teradata supports Aggregation pushdown for below specified cases.