Replies: 1 comment
-
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
We have rewritten many AQE test cases with hard-coded Spark configurations and judging conditions in each test case, because Gluten generates shuffle output of different sizes compared to the statistics used by AQE. Currently, the CH backend and the Velox backend share the Spark UT modules, thereby sharing the AQE test cases as well. Since different backends can generate varying shuffle statistics, we may need to separate the AQE Spark UTs. This separation will allow us to modify and use different AQE configurations for each backend individually.
So far, I found 2 test suites that need to be separated:
To address this issue, we can create packages
org.apache.spark.sql.execution.adaptive.veloxandorg.apache.spark.sql.execution.adaptive.clickhouseand place the Suites for CH/Velox under these packages.Beta Was this translation helpful? Give feedback.
All reactions