Estimate sensitivity and specificity

Can iMRMC estimate sensitivity and specificity?

Yes! The iMRMC can estimate sensitivity and specificity. However, the key is how to format the data as the input file. Your should starting with ROC data, which means you have disease and non-disease cases, and the threshold for sensitivity and specificity. The first 3 columns are the same as the input for AUC analysis of ROC data. The last column now represents the zero-one (negative-positive) binary decision.

Transform iMRMC input file for Sensitivity or Specificity Analysis

1. Do not need to edit truth data

2. Edit score data

Analysis Purpose	Case Truth	Reader decision	Binary decision
Sensitivity	Disease case	Disease (score > threshold)	1
Sensitivity	Disease case	Non-disease (threshold > score)	0
Sensitivity	Non-disease case	Does not matter	0.5
Specificity	Disease case	Does not matter	0.5
Specificity	Non-Disease case	Disease (score > threshold)	1
Specificity	Non-disease case	Non-disease (threshold > score)	0

3. Example

Calculate pathologists’ sensitivity for dataset A

• For disease cases, if reader’s score is larger than or equal to the threshold (≥3), set the last column for that reader and case to 1 (e.g. since reader2 gave score 4 to case3, set the last column of this row to 1)
• For disease cases, if reader’s score is lower than threshold (<3), set the last column for that reader and case to 0 (e.g. since reader11 gave score 1 to case3, set the last column of this row to 0)
• Set all scores (last column) for non-diseases cases to 0.5 (e.g. since case1 is non-disease case, set last column for case1 from all readers to 0.5).

Note

In GUI and output files, the results are still displayed as AUC but not sensitivity and specificity.