[Spark 4.1.0] TimeAdd expression renamed to TimestampAddInterval

[res-life]

Summary

In Spark 4.1.0, the TimeAdd expression was renamed to TimestampAddInterval.

Details

• Spark Version: 4.1.0
• Change Type: Class rename

Spark ≤4.0.x

package org.apache.spark.sql.catalyst.expressions
case class TimeAdd(start: Expression, interval: Expression, ...)

Spark 4.1.0+

package org.apache.spark.sql.catalyst.expressions
case class TimestampAddInterval(start: Expression, interval: Expression, ...)

Impact

Code that references TimeAdd directly will fail to compile against Spark 4.1.0:

not found: type TimeAdd

Affected Files

• sql-plugin/src/main/spark320/scala/com/nvidia/spark/rapids/shims/TimeAddShims.scala
• sql-plugin/src/main/spark330/scala/com/nvidia/spark/rapids/shims/DayTimeIntervalShims.scala
• sql-plugin/src/main/scala/org/apache/spark/sql/rapids/shims/GpuTimeAdd.scala

Solution

Create version-specific shims:

For Spark 3.2.0-4.0.x (spark320/ and spark330/):

• Use TimeAdd expression in TimeAddShims.scala

For Spark 4.1.0+ (spark410/):

• Create empty TimeAddShims.scala (returns empty map)
• Handle TimestampAddInterval in DayTimeIntervalShims.scala with GpuTimestampAddInterval

References

• Spark 4.0.1: TimeAdd exists
• Spark 4.1.0: Renamed to TimestampAddInterval

[res-life]

I'll finish the shim and make Spark 410 shim building pass, but need more investigations about potential impacts, tests.

[res-life]

Will be closed by #14120