Skip to content

LoadFileMixin always opens the whole file #459

@jmilloy

Description

@jmilloy

Description

The LoadFileMixin is used by rasterio, csv, and xarray nodes for both local and remote files. As written, the it always loads the whole file. The Mixin should be updated to only load the whole file in certain cases, and otherwise open the dataset directly from the source path.

Describe the solution you'd like

Distinguish between local and remote sources. Only read the file into memory if necessary for caching.

Additional Notes

The Rasterio node should be modified to not use the mixin. This will make the read_from_source unnecessary.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions