iceberg failed to build with datafusion-comet 0.11.0 in JDK 11

[manuzhang]

Describe the bug

As in apache/iceberg#14591, Iceberg build in JDK 11 failed after upgrading to datafusion-comet.

> Task :iceberg-spark:iceberg-spark-3.5_2.12:compileScala
javac exited with exit code 1
Error:  /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:23:32:  error: cannot access AbstractColumnReader
Error:  /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:24:32:  error: cannot access ColumnReader
Error:  /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:25:32:  error: cannot access TypeUtil
Error:  /home/runner/work/iceberg/iceberg/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java:26:32:  error: cannot access Utils

It looks to be broken by #2514

cc @parthchandra @andygrove

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

[parthchandra]

I don't think we can upgrade to Comet 0.11.0 unless apache/iceberg#13786 is merged.
FWIW, iceberg with the aforementioned PR builds successfully with Comet 0.11.0 for all versions of Spark in my local build. I did use JDK 17 though because Spark 4 support requires it.

[manuzhang]

@parthchandra The error was thrown when building iceberg-spark-3.5 and iceberg-spark-4.0 is built with JDK17+. I see apache/iceberg#13786 is required, but the build should pass at least.

[parthchandra]

TBH, I think it is better if we do not update Comet in Iceberg until the PR-13786 is merged. Without it, Comet-Iceberg is broken and updating Comet version simply gives the impression that all is fine.

[manuzhang]

@parthchandra That makes sense. I will rebase the PR after apache/iceberg#13786 is merged. I'm resolving and asking about the build issue in JDK11.

[parthchandra]

Tried with jdk 11 and you're right the build does fail. It succeeds with jdk 17. I don't see an obvious reason for the failure.

[manuzhang]

Iceberg build can pass with datafusion-comet 0.12.0 with removal of JDK 11.