Generate some data for regression tests

[davemfish]

As we refactor the R code into python, we should have a suite of tests to make sure the python version is producing the same results as in R.

For the knn_bootstrap method & precipitation, the raw results (predicted values) and some summary stats are stored in,
<model>_<scenario>_pr_<n>_bootstrap.csv
<model>_<scenario>_pr_stats.csv

For heatSignal method & temperature, there are these two options, I'm not sure which is best:
<model>_<scenario>_tas_heatSignal.csv
<model>_<scenario>_TS_HeatSignal_WP.csv

These data can be the basis for regression tests of downscaling methods.

• check-in R code
• check-in other configuration/setup files that define input parameters
• run R code for the region already setup in our shared drive (or perhaps a smaller region)
• check-in key CSV tables of results
• add a tag to this repo so it's easy to find the version that generated the regression data

[flector-co]

Thanks @davemfish. I agree with the overall idea. Here some comments regarding how to implement the process:

• About the knn_bootstrap:

The key output is <model>_<scenario>_pr_<n>_bootstrap.csv

As this is a stochastic method, is not necessarily possible to test the method by comparing directly the predicted values. There are two places where the method uses random numbers: 1. To make the selection of the transitions of next system state (i.e. that determine of the next day will be dry, wet, etc), and 2. to choose one day in the historical record that has a similar transition.

I see two options to design the test. We could lock the seed of the random numbers that the algorithm uses (I wonder if can be done between R and Python?, or maybe just feeding a fixed a series of uniformly distributed numbers). The second would be to use a statistical method to compare the results (i.e. calculate the similarity of seasonal percentiles, rather than actual values).

• About the heatSignal:

The result output to compare is <model>_<scenario>_tas_heatSignal.csv

[davemfish]

Thanks, @flector-co , great points about the stochastic nature of method. I think either option for testing could make sense. We'll see what is most convenient as we go.