You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+45-43Lines changed: 45 additions & 43 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -55,6 +55,51 @@ result
55
55
-**Cross-validation:** Counterfactual-aware model selection with `cf_cv()` and `cf_compare()`
56
56
-**Transportability:** Evaluate model performance when transporting from a source population (e.g., RCT) to a target population
57
57
58
+
## Counterfactual Performance Estimation
59
+
60
+
Estimate how a prediction model would perform under a hypothetical treatment policy (e.g., if everyone received or avoided treatment):
61
+
62
+
```r
63
+
library(cfperformance)
64
+
data(cvd_sim)
65
+
66
+
# Compare estimators for counterfactual MSE
67
+
estimators<- c("naive", "cl", "ipw", "dr")
68
+
sapply(estimators, function(est) {
69
+
cf_mse(
70
+
predictions=cvd_sim$risk_score,
71
+
outcomes=cvd_sim$event,
72
+
treatment=cvd_sim$treatment,
73
+
covariates=cvd_sim[, c("age", "bp", "chol")],
74
+
treatment_level=0,
75
+
estimator=est
76
+
)$estimate
77
+
})
78
+
79
+
# Estimate counterfactual AUC
80
+
cf_auc(
81
+
predictions=cvd_sim$risk_score,
82
+
outcomes=cvd_sim$event,
83
+
treatment=cvd_sim$treatment,
84
+
covariates=cvd_sim[, c("age", "bp", "chol")],
85
+
treatment_level=0,
86
+
estimator="dr"
87
+
)
88
+
89
+
# Cross-validation for model selection
90
+
cf_compare(
91
+
models=list(
92
+
"Simple"=event~age,
93
+
"Full"=event~age+bp+chol
94
+
),
95
+
data=cvd_sim,
96
+
treatment="treatment",
97
+
treatment_level=0,
98
+
metric="mse",
99
+
K=5
100
+
)
101
+
```
102
+
58
103
## Transportability Analysis
59
104
60
105
The package also implements transportability estimators from Voter et al. (2025) for evaluating prediction model performance when transporting from a source population (typically an RCT) to a target population:
@@ -132,49 +177,6 @@ cf_mse(
132
177
133
178
See `vignette("ml-integration", package = "cfperformance")` for details.
134
179
135
-
## Example
136
-
137
-
```r
138
-
library(cfperformance)
139
-
data(cvd_sim)
140
-
141
-
# Compare estimators
142
-
estimators<- c("naive", "cl", "ipw", "dr")
143
-
sapply(estimators, function(est) {
144
-
cf_mse(
145
-
predictions=cvd_sim$risk_score,
146
-
outcomes=cvd_sim$event,
147
-
treatment=cvd_sim$treatment,
148
-
covariates=cvd_sim[, c("age", "bp", "chol")],
149
-
treatment_level=0,
150
-
estimator=est
151
-
)$estimate
152
-
})
153
-
154
-
# Estimate counterfactual AUC
155
-
cf_auc(
156
-
predictions=cvd_sim$risk_score,
157
-
outcomes=cvd_sim$event,
158
-
treatment=cvd_sim$treatment,
159
-
covariates=cvd_sim[, c("age", "bp", "chol")],
160
-
treatment_level=0,
161
-
estimator="dr"
162
-
)
163
-
164
-
# Cross-validation for model selection
165
-
cf_compare(
166
-
models=list(
167
-
"Simple"=event~age,
168
-
"Full"=event~age+bp+chol
169
-
),
170
-
data=cvd_sim,
171
-
treatment="treatment",
172
-
treatment_level=0,
173
-
metric="mse",
174
-
K=5
175
-
)
176
-
```
177
-
178
180
## Documentation
179
181
180
182
See `vignette("introduction", package = "cfperformance")` for a comprehensive introduction.
0 commit comments