[VL] Support read old ORC file without column names#8862
[VL] Support read old ORC file without column names#8862ccat3z wants to merge 6 commits intoapache:mainfrom
Conversation
|
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format? See also: |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
cc @kecookier |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. |
|
Cc: @rui-mo |
|
Run Gluten Clickhouse CI on x86 |
|
@rui-mo Could you rebase this PR ? Thanks. |
|
@ccat3z Thank you for the reminder. I will pick and check the function. |
|
@ccat3z may I ask what's the state of currrent PR, do you plan to finish it? |
|
@beliefer @taiyang-li Would you like to create an issue to track this problem? As @ccat3z noted, the pull request #10697 was aimed at supporting reading ORC files by matching indices, but there may still be a gap for the scenario you described. Let’s continue the discussion in a dedicated issue. Thanks. |
|
@rui-mo @taiyang-li @ccat3z I already created #11010. |
|
Fixed in #10697 |
What changes were proposed in this pull request?
An ORC file written by an old version has no field names in the physical schema. To read it, we must map table schema to file schema using indices.
ScanTransformer#getDataColumnsas table schema to Velox.This PR depends on facebookincubator/velox#12489 (old ORC files) and facebookincubator/velox#12490 (match index mapping behavior in spark).
Fixed #5638.
How was this patch tested?
Unit tests.