We've seen a case of a user fitting a cov_adj model with the treatment variable in it, but cov_adj() not identifying that variable as a treatment variable, and so not zeroing out its contributions to the predictions. The issue was that in the cov_adj model the treatment had a different name.
As it happens, the variable identified as treatment in the StudySpecification did show up in the newdata frame that was produced in the course of the call to cov_adj(), and so that variable's variation did get appropriately flattened by the steps at R/cov_adj.R#L68-L79. It's just that there was another variable in there that cov_adj() didn't know to flatten but that ought to have been flattened.
This user says a warning would have been helpful to her for identifying or heading off this problem:
In my case, the trt_name from the specification was indeed in newdata, but it was not in model. So, I would add a check as to whether trt_name is a term in the model, and if not, add a warning along the lines of "trt_name from specification is not a term in the model specified. Was the model fit solely to control data? If not, make sure to use the same treatment variable name in the model and in the specification so adjustment can be made appropriately under the control condition."
A warning like this wouldn't be appropriate if only control observations had contributed to the fit of the covariance model. Or just treatments -- more generally, if in the intersection of the Q and C samples, the StudySpecification identified treatment variable takes at most one value on the Q sample. At the end of cov_adj(), we produce a SandwichLayer object, which has in its @keys data frame an in_Q column that will tell us if the row is in the Q sample. In uses cases like the one prompting this issue, earlier on at L68-L79 the StudySpec identified treatment variable was found to be in newdata. So it seems that at that moment we have the elements needed to perform a check along the lines requested.
We've seen a case of a user fitting a cov_adj model with the treatment variable in it, but cov_adj() not identifying that variable as a treatment variable, and so not zeroing out its contributions to the predictions. The issue was that in the cov_adj model the treatment had a different name.
As it happens, the variable identified as treatment in the
StudySpecificationdid show up in thenewdataframe that was produced in the course of the call tocov_adj(), and so that variable's variation did get appropriately flattened by the steps at R/cov_adj.R#L68-L79. It's just that there was another variable in there that cov_adj() didn't know to flatten but that ought to have been flattened.This user says a warning would have been helpful to her for identifying or heading off this problem:
A warning like this wouldn't be appropriate if only control observations had contributed to the fit of the covariance model. Or just treatments -- more generally, if in the intersection of the Q and C samples, the
StudySpecificationidentified treatment variable takes at most one value on the Q sample. At the end ofcov_adj(), we produce aSandwichLayerobject, which has in its@keysdata frame anin_Qcolumn that will tell us if the row is in the Q sample. In uses cases like the one prompting this issue, earlier on at L68-L79 theStudySpecidentified treatment variable was found to be innewdata. So it seems that at that moment we have the elements needed to perform a check along the lines requested.