Skip to content

[GLUTEN-10207][VL] adding spark40 unit test framework#10725

Merged
zhouyuan merged 61 commits intoapache:mainfrom
zhouyuan:wip_spark400_unittests
Nov 14, 2025
Merged

[GLUTEN-10207][VL] adding spark40 unit test framework#10725
zhouyuan merged 61 commits intoapache:mainfrom
zhouyuan:wip_spark400_unittests

Conversation

@zhouyuan
Copy link
Member

@zhouyuan zhouyuan commented Sep 16, 2025 •
edited
Loading

What changes are proposed in this pull request?

This patch adds the spark unit test framework for Spark-400. The failed unit tests are ignored firstly.
The main gaps are on

  • Velox backends: backends-velox/src/test. There are several tests failed due to plan check different
  • src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala: ignored several suite, mostly on CSVSuite, SQLQuerySuite, ANSISuite, ArrowPythonEval

all failed unit tests/suites are marked with // TODO: fix in Spark-4.0

Note there are some CK backend related tests also added, this will not impact CK backend as it does not support Spark-4.0 yet.
related:
#11088

How was this patch tested?

Pass GHA

Related issue: #10207

@github-actions github-actions bot added the CORE works for Gluten Core label Sep 16, 2025
@github-actions
Copy link

github-actions bot commented Sep 16, 2025

#10207

@github-actions
Copy link

github-actions bot commented Sep 16, 2025

Run Gluten Clickhouse CI on x86

@github-actions github-actions bot added the INFRA label Sep 16, 2025
@github-actions
Copy link

github-actions bot commented Sep 16, 2025

Run Gluten Clickhouse CI on x86

6 similar comments
@github-actions
Copy link

github-actions bot commented Sep 16, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link

github-actions bot commented Sep 16, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link

github-actions bot commented Sep 29, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link

github-actions bot commented Sep 29, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link

github-actions bot commented Sep 29, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link

github-actions bot commented Sep 29, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan marked this pull request as ready for review September 29, 2025 13:17
@github-actions
Copy link

github-actions bot commented Sep 29, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Sep 29, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link

github-actions bot commented Oct 2, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Oct 2, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark400_unittests branch from c03c388 to 2d2119f Compare October 2, 2025 18:45
@github-actions
Copy link

github-actions bot commented Oct 2, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Oct 2, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark400_unittests branch from a557bb0 to a7e6144 Compare October 2, 2025 20:57
@github-actions
Copy link

github-actions bot commented Oct 2, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Oct 6, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark400_unittests branch from ec28c41 to 2af153e Compare October 31, 2025 13:49
@github-actions
Copy link

github-actions bot commented Oct 31, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Oct 31, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark400_unittests branch from 20f6d8e to ab044df Compare October 31, 2025 14:48
@github-actions
Copy link

github-actions bot commented Oct 31, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Oct 31, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link

github-actions bot commented Nov 11, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
ignore more ut
ae3ab79 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
fix month_between
bbaf177 
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions
Copy link

github-actions bot commented Nov 11, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
ignore more ut
8ab2ba7 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan force-pushed the wip_spark400_unittests branch from 6a67f66 to 8ab2ba7 Compare November 11, 2025 20:46
@github-actions
Copy link

github-actions bot commented Nov 11, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
ignore more ut
288768b 
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions
Copy link

github-actions bot commented Nov 11, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Nov 12, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
ignore more ut
c03d5e8 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan force-pushed the wip_spark400_unittests branch from 28b6bda to c03d5e8 Compare November 12, 2025 13:44
@github-actions
Copy link

github-actions bot commented Nov 12, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan changed the title [GLUTEN-10207][VL] adding spark40 unit tests [GLUTEN-10207][VL] adding spark40 unit test framework Nov 12, 2025
@zhouyuan
fix junit version
9a3aeb1 
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions
Copy link

github-actions bot commented Nov 12, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
fix
1fb85c1 
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions
Copy link

github-actions bot commented Nov 12, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
fix
16faa57 
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
ignore csv ut
70e8913 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan force-pushed the wip_spark400_unittests branch from 6ec699d to 70e8913 Compare November 13, 2025 13:26
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan requested a review from PHILO-HE November 13, 2025 23:06
@zhouyuan zhouyuan mentioned this pull request Nov 14, 2025
Open
@zhouyuan zhouyuan merged commit 2864633 into apache:main Nov 14, 2025
112 of 113 checks passed
@zhouyuan
Copy link
Member Author

zhouyuan commented Nov 14, 2025

@marin-ma Thank you for the check!

@baibaichen baibaichen mentioned this pull request Dec 29, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@marin-ma marin-ma marin-ma approved these changes

@PHILO-HE PHILO-HE Awaiting requested review from PHILO-HE

Assignees

No one assigned

Labels

BUILD CORE works for Gluten Core INFRA VELOX

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@zhouyuan @marin-ma