[GLUTEN-4889][VL] feat: Support approx_percentile aggregate function

[Yizhou-Yang]

What

Add Velox approx_percentile support for Spark.

Why

Velox uses KLL sketch while Spark uses GK algorithm — their intermediate data formats are incompatible (KLL: 9-field StructType vs GK: single BinaryType buffer). This means fallback between Velox and Spark requires separate handling.

How

• VeloxApproximatePercentile: A DeclarativeAggregate with 9 aggBufferAttributes matching Velox's KLL sketch layout.
• Spark-side KLL implementation (KllSketchHelper/KllSketchAdd/KllSketchMerge/KllSketchEval): Simplified KLL operations for fallback, binary-compatible with Velox's C++ accumulator.
• ApproxPercentileRewriteRule: Rewrites Spark's ApproximatePercentile to the Velox-compatible version.
• All 4 fallback modes supported: Full offload, partial fallback, final fallback, full fallback.

Key decisions

• Accuracy stored as IntegerType (Spark's original value); Velox computes epsilon = 1.0/accuracy internally.
• KLL chosen over GK for Spark-side fallback to maintain intermediate data compatibility with Velox.

Velox dependency

facebookincubator/velox#16320

---

Related issue: #4889

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[jinchengchenghh]

Please update get-velox.sh to test your PR, then you can verify if both can work well, you may update this line

incubator-gluten/ep/build-velox/src/get-velox.sh

Line 28 in 5d3f714

UPSTREAM_VELOX_PR_ID=""

[jinchengchenghh]

Do we need the config? Usually we offload the function to native by default

[Yizhou-Yang]

Please update get-velox.sh to test your PR, then you can verify if both can work well, you may update this line

https://github.com/apache/incubator-gluten/blob/5d3f7145cd7fc258aa10b434ea4ec651bd82c764/ep/build-velox/src/get-velox.sh#L28

added the 16320 and removed the config

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[jinchengchenghh]

Please update the PR description to describe the KLL Sketch is different so that we handle fallback separately.

[Yizhou-Yang]

Please update the PR description to describe the KLL Sketch is different so that we handle fallback separately.

done~

[github-actions]

Run Gluten Clickhouse CI on x86

[Yizhou-Yang]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[Yizhou-Yang]

There are test failures in x86 environment non-slow modes.

edit:
The problem isn't that simple. it is related to gk vs kll. I might need to exclude/rewrite tests.
I will try to keep the original tests, but approximate algorithm does not give a definitive result sometimes.
I need more time on this

[jiangjiangtian]

@Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the accuracy requirement. Will the relative error rate be too high?

[Yizhou-Yang]

@Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the accuracy requirement. Will the relative error rate be too high?

yes, this is part of the reason for the ut failures. The gluten version of kll is written rather hastily and was primarily concerned with simplicity and fallback compatibiltity with velox, and now it seems to have caused problems including but not limited to ut failures and lots of off-by-one issues. I am editing this implementation and doing a line-by-line comparison with the velox one to ensure the gluten side quality, and enhanced unit tests needs to be performed. Although after all this, there will still be some existing uts that might need to be excluded.