Skip to content

check existing data before downloading new data #61

@kimlee87

Description

@kimlee87

Since data from ERA5-Land is updated with a specific interval, it makes sense to just download the newly updated data and not re-download the old data.

For example: if users already downloaded data from 2021 to 2023, now they trigger a query to download data from 2021 to 2024, will the package alert user about existing data?

Inital thought:

  • check for raw data (i.e. downloaded data) in data lake to find out existing periods
    • This requires consistent naming across files at all time (?)
  • check for data stored in database
    • only for monthly data for now

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions