Skip to content

Distributed SQL query with Burla.dev's remote_parallel_map. #92

@alxmrs

Description

@alxmrs

Check out Burla's 1 Trillion Row Challenge demo: https://docs.burla.dev/examples/process-2.4tb-in-parquet-files-in-76s

It seems like with this library as it is today, we could use Burla's remote_parallel_map and perform highly scalable, efficient distributed SQL processing on Xarray data.

I think this could help us get this project to the 100 Trillion row challenge (#34) in a really short amount of time.

CC: @JacobZuliani, @jayendra-info.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions