.check_df_moderator_estimates

[jwasserman2]

In .check_df_moderator_estimates(), our function for heuristically checking whether there are sufficient degrees of freedom to estimate a SE for a moderator effect estimate, at line 512 rather than having:

propertee/R/SandwichLayerVariance.R

Line 512 in bb862c6

in_model_fit <- rep(1, length(cluster))

we should have a line where elements of in_model_fit are 0's if the associated observation has a weight of zero.

[jwasserman2]

Labeling this as discussion to provide a space to discuss the y~0 formula functionality and the use of the main effects estfun instead of the subgroup effects estfun in situations with insufficient degrees of freedom. Two questions, @benthestatistician, when you have a chance:

1. Can you explain your vision for how y~0 works? That's not a formula the user passes to lmitt() but rather a formula we update to in order to inform the lmitt() fit?
2. .check_df_moderator_estimates() is currently called after the bread and meat matrices have been computed. Am I right in thinking we would want to run .check_df_moderator_estimates() before those computations to inform whether we would want to replace the subgroup effects estfun with the main effects estfun for a given fit?

[benthestatistician]

To answer in order reflecting the priority of the two notions I had floated to @jwasserman2 elsewhere, and that prompted these discussion points:

2. My idea was less that we fiddle with the current scheme, more that after that scheme had kicked in, we equip users to kick off an alternate variance calculation, one that figures estfun's and meat by setting all of the subgroup treatment effect parameters to the parameter value estimated for the main effect model. (It wouldn't need to fiddle with the A matrices, if I'm not mistaken.)
1. ("vision" probably isn't the right word here) I would like a way to estimate the treatment effect model with intercept and other ancilliary parameters free but the treatment effect (or effects) set to a fixed value. (One does this for example in score testing of the hypothesis that the pre-set value of the interest parameter is the correct one.) In an ordinary lm() setting one does this by moving the treatment term from the right hand side of the regression specification to the offset= postion, after multipling it by the hypothesized value.
But lmitt() includes a treatment effect by default, with currently no means of overriding. I had at one point stuck up for +0 as a way to mimic the lm() approach sketched immediately above, but I'm not sure that achieves everything we'd want it to. So I'd think first about (2), the higher priority of these two lower priority items, and then think about whether what makes sense for (2) might also offer some help with (1).

Connecting these points back to the to-do stated at the top of the issue: I don't presently see (1) or (2) as a blocker or complicator of the proposed improvement to the .check_df_moderator_estimates() logic.

[jwasserman2]

Also noting that our current policy of re-defining cluster ID's for small strata to be the stratum ID isn't implemented here. Fixing that should be part of work on this issue

[jwasserman2]

To accommodate a values_from argument passed from vcov_tee, apply the df check only to estimates in the overlap of the x and values_from arguments of vcov_tee (with values_from defaulting to x in `vcov_tee)

[jwasserman2]

Observations with zero weight contributing to degrees of freedom has been addressed in 07dac97.

Degrees of freedom counts not aligning for model-based vcov estimates in settings with small blocks has been addressed in de1aa3f.

Checking degrees of freedom for moderators that weren't included in the values_from fit has been addressed in 1fe28e3.

These commits are bundled in PR #260.