Skip to content

Break up processing into chunks #68

@gutmann

Description

@gutmann

To permit larger runs in limited memory it would be nice to be able to specify a chunk size in time (or space) over which GARD would process datasets.

Performing this split in time might be the most obvious (and goes along with splitting up the output files #11 )

Performing the split in space could be more efficient because GARD needs a complete time series for each grid point in the training period regardless of how much time is being processed. Would this end up writing separate files for each subset that need to be recombined later or could it write just the subset it processed to an output file that contains the entire grid?

Ideally both should be options, but do they need to be?

Tradeoffs in implementation complexity and benefit should be assessed briefly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions