Modules

[emhuff]

A. Config generator:

• has a (long) list of optional keyword arguments, all of which have sensible defaults. Outputs a config file.

B. Data generator

• eats a config file, returns a data vector with added systematics and noise according to the model

C. Fitter

• eats a data vector and config file, fits parameters; returns an updated config file with new parameter values and ranges (we should decide how to record uncertainties)

D. Tension metric

• eats two config files, two data vectors, returns two chi-squares and one of several tension metrics describing a distance between the experimental results

E. Decider

• eats the chi-squareds, config files, and tension metrics. Returns a pair of updated config files; where tension and goodness-of-fit metrics have determined whether and how models have been updated; see Rules to decide about changing #4 .

[bnord]

@emhuff Have we covered all of this? What's next in our plan?

[emhuff]

Once we're satisfied that with a sufficiently interesting combination of consensus decision rules, and we think the fitters are working properly, I think there are two broad tasks that have to be done next:

1. We need to have a transparent, automated pipeline for running simulations and generating and storing consensus results. We've started on this, but key elements include a config interface that fully specifies a run such that, with the config in hand, you can exactly reproduce a previous result.

2. The harmonic oscillator experiment classes were originally conceived as a toy model, showing readers what we mean by "consensus" and how the decision rules operate. It would be really nice to be able to do the same exercise for a more realistic cosmology case -- we'd talked about using measurements of the growth function and expansion history at different redshifts (e.g., H(z) or angular diameter distance) as the data vector, and then keeping the rest of the machinery as is. I think this only really requires writing new cosmology classes, which we ought to be able to do without introducing any other dependencies beyond astropy.cosmology.