Skip to content

[GLUTEN-1433] [VL] Add config to disable TimestampNTZ validation fallback#11720

Open
Mariamalmesfer wants to merge 1 commit intoapache:mainfrom
Mariamalmesfer:validate-localtimestamp
Open

[GLUTEN-1433] [VL] Add config to disable TimestampNTZ validation fallback#11720
Mariamalmesfer wants to merge 1 commit intoapache:mainfrom
Mariamalmesfer:validate-localtimestamp

Conversation

@Mariamalmesfer
Copy link
Contributor

@Mariamalmesfer Mariamalmesfer commented Mar 8, 2026

This PR adds a config to control fallback validation for TimestampNTZType in the Velox backend and adds a test for localtimestamp().

Currently, the validator treats TimestampNTZType as unsupported and forces the query to fall back to Spark. This makes it hard to develop and test features related to TimestampNTZ, including functions like localtimestamp().

With this change, the validation rule can be temporarily disabled during development and testing.

Related issue: #1433

@github-actions github-actions bot added the VELOX label Mar 8, 2026
@Mariamalmesfer Mariamalmesfer force-pushed the validate-localtimestamp branch 3 times, most recently from ae23e8b to 15b228c Compare March 9, 2026 20:13
@github-actions github-actions bot added the DOCS label Mar 9, 2026
@Mariamalmesfer Mariamalmesfer force-pushed the validate-localtimestamp branch 5 times, most recently from 14ab5a6 to 0b12c90 Compare March 9, 2026 22:19
@Mariamalmesfer Mariamalmesfer changed the title Add validation test for localtimestamp [GLUTEN-1433] [VL] Add config to disable TimestampNTZ validation fallback for Velox development Mar 10, 2026
@Mariamalmesfer Mariamalmesfer changed the title [GLUTEN-1433] [VL] Add config to disable TimestampNTZ validation fallback for Velox development [GLUTEN-1433] [VL] Add config to disable TimestampNTZ validation fallback Mar 10, 2026
@Mariamalmesfer Mariamalmesfer force-pushed the validate-localtimestamp branch from 0b12c90 to e91e760 Compare March 10, 2026 11:50
@Mariamalmesfer Mariamalmesfer marked this pull request as ready for review March 10, 2026 11:50
case BooleanType | ByteType | ShortType | IntegerType | LongType | FloatType | DoubleType |
StringType | BinaryType | _: DecimalType | DateType | TimestampType |
YearMonthIntervalType.DEFAULT | NullType =>
true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also check this flag in the validator code here

private class FallbackByTimestampNTZ() extends Validator {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants