Skip to content

how to determine if new data has been added #16

@dcarver1

Description

@dcarver1

Need to refine this questions but here the main idea.

How does an analysis determine if they need to rerun all function and push an updated resource to the public site.
Two means
-1. test for updates on all individual sources

    1. run the processes and test for difference between current and previous output

I only want to apply option one for datasets that are updated manually. When you do that just rerun the process and update data on the application

If the data processing is fast enough, I'd suggest running the data processing code once every month and using something described here
https://stackoverflow.com/questions/43081791/in-r-find-whether-two-files-differ/53333988 answer by stephanmg
to determine if the new output file is different from the previous, if so the new file is save if not the previous file remains and now updates are needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions