feat: Add IcebergDataSource#16177
Conversation
✅ Deploy Preview for meta-velox canceled.
|
e6ac988 to
25a402c
Compare
|
Hi @PingLiuPing, there are some build failures in CI. Could you please take a look if they are relevant to this change? Or rebase onto the latest main if they're not. Thanks! |
25a402c to
9df83bd
Compare
@kagamiori Thank you, rebased and updated the code. |
9df83bd to
2331e0f
Compare
|
@kagamiori has imported this pull request. If you are a Meta employee, you can view this in D92066506. |
|
@kagamiori Could you help check the internal tests? It looks like they’ve been running for about a day. |
|
@kagamiori merged this pull request in 3177849. |
This reverts commit 3177849.
Summary: Add IcebergDataSource to support creating IcebergSplitReader instances, and remove the IcebergSplitReader creation logic from the Hive split reader. IcebergSplitReader is the last remaining Iceberg-specific symbol that Hive depends on. By moving this logic out of Hive, Hive no longer has a dependency on Iceberg. As a result, the Hive CMake configuration can be simplified and cleaned up. Pull Request resolved: facebookincubator#16177 Reviewed By: kKPulla Differential Revision: D92066506 Pulled By: kagamiori fbshipit-source-id: 15c4b46d36882eca366cacdb30659491b92ade59
This reverts commit 3177849.
|
This PR breaks Iceberg position delete read, after this PR, the tests failed in Gluten, VeloxIcebergSuite failed by https://github.com/apache/incubator-gluten/actions/runs/21911829182/job/63418832448?pr=11587 Could you help find why the test failed? Thanks! Is it because we should update Gluten code to use IcebergConnectorFactory? @PingLiuPing |
Add IcebergDataSource to support creating IcebergSplitReader instances, and remove the IcebergSplitReader creation logic from the Hive split reader.
IcebergSplitReader is the last remaining Iceberg-specific symbol that Hive depends on. By moving this logic out of Hive, Hive no longer has a dependency on Iceberg. As a result, the Hive CMake configuration can be simplified and cleaned up.