Calculation of OR parameters from summary info without reader AUCs

[cardypen]

I have another question relating to the iMRMC Java application. Do you know how the app handles the case where summary information is provided, but individual reader AUCs are not specified? The app runs when only moments and average AUCs for each modality are specified, but I am wondering how the OR parameters are calculated without the reader AUCs or if there is a default value that I’m missing. I would appreciate any direction you’re able to give. Thanks again for your time!

[brandon-gallas]

I regret that we are no longer supporting the iMRMC Java Application. It has been a long time since I have touched that code. No promises on what I say. It would be better for you to use the R package and code.

Here is an example "Summary Info" file

• MRMCSummary.txt
• I can see a section on Reader Specific AUCs.
• You imply that the Reader Specific AUCs are not specified, but you can still run the sizing app.

The MRMC variance of AUC can be decomposed in many ways. My original method used 8 product moments of the success outcome. In this paper Gallas2009_Commun-Stat-A-Theor_v38p2586, I derived the relationships between the following:

• Appendix A Success moments and the variance components derived by Barrett, Clarkson, and Kupinski.
• Appendix C Success moments and the DBM variance components.
• The ingredients to deriving the OR parameters in terms of the product moments of the success outcomes are all there.

Note that the 8th success moment == (reader-averaged AUC)^2. This implies that you don't need the individual-reader AUCs to derive the variance components. You only need the reader-averaged AUC. A "default value" is not needed.

Hillis derived the relationship between the OR variance components and my methods

• Title: Relationship between Obuchowski–Rockette–Hillis and Gallas methods for analyzing multi-reader diagnostic imaging data with empirical AUC as the reader performance measure
• Hillis2023_J-Biostat-Epidemiol_v7pe2062115
• I was invited to be a coauthor on this paper, but I couldn't follow the notation.

[brandon-gallas]

I should also mention that this document contains a sizing analysis with corresponding code. It's how I sized a large study of my own.

• Viper Data Repo
• R Markdown File
• R Markdown File Rendered to PDF

[cardypen]

Thank you very much for taking the time to respond. I really appreciate it. I will go through the references you have shared.

Our goal with this work is to create a user-friendly way to size MRMC studies in settings where pilot data are not available, using only Obuchowski–Rockette style parameters. In our experience, these parameters, such as ranges of between- and within-reader variability, are much more accessible and interpretable for clinicians and imaging collaborators than other parameterizations.

The iMRMC Java application is the closest existing tool we have found that supports this goal, since it allows study sizing when provided with AUCs and moment information rather than only raw reader scores. Much of our effort has therefore been focused on understanding how to map OR-style assumptions into the quantities required by iMRMC in a principled and transparent way.

We also share your perspective regarding the complexity of the notation in this area. One of our motivations is to help unify and clarify the various formulations, so that the underlying ideas are easier to digest for applied researchers while remaining faithful to the established theory.

Thank you again for your time and for engaging with this question. Your feedback is very helpful as we refine this work.