Add a new HoodieHFileReader for Trino with Java 11 #14979
Description
To address compatibility issues of HBase 1.2.3 with Java 11 for Hudi Trino connector
JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3127
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-2687
Comments
03/Jan/22 06:36;guoyihua;I tried two routes and hit issues:
(1) Add a new HudiHFileBootstrapIndex and a config for the meta client to use that for bootstrap index in Trino repo ([https://github.com//pull/4494/files], [https://github.com/codope/trino/pull/11/files]). It hits the compatibility issue due to the comparator {{Caused by: java.lang.ClassCastException: class org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex$HoodieKVComparator cannot be cast to class org.apache.hadoop.hbase.CellComparator}} [https://gist.github.com/yihua/f14d8edd3c876838f02a5e57e90ea4b0]. It looks like the custom comparator to work around shading causes this. Built-in KVComparator/CellComparators have its backward class name translation in {{FixedFileTrailer}} class
(2) Use HBase 1.7.1 at runtime for Trino Hudi connector ([https://github.com/codope/trino/pull/12]). Hitting another issue of guava: {{java.lang.NoSuchMethodError: 'com.google.common.base.Objects$ToStringHelper com.google.common.base.Objects.toStringHelper(java.lang.Object)'}} [https://gist.github.com/yihua/e3bc8ebc36aa0fc04127322677bde55a];;;
03/Jan/22 07:20;guoyihua;For the first set of changes, I also need to copy hbase-shaded-protobuf-4.0.1.jar to the hudi plugin folder for testing. I cannot address the dup proto resources at this moment.;;;