Skip to content

[Flink][Enhance] Velox should return empty row if no records consumed from data source like kafka #29

@KevinyhZou

Description

@KevinyhZou

Backend

VL (Velox)

Bug description

Flink从 流式数据源中消费数据,如kafka,如果kafka topic中没有数据,此时应该返回一个Empty row。

Velox 只考虑到了批处理读取数据的情况,在Velox 的TableScan中如果 返回empty row,会一直循环去读取,直到读取到有数据的vector返回。

此处对应流计算的场景来说是不合理的,此时应该返回empty row,给上层flink。

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions