Skip to content

PMXRD-219 Quantitative Predictive Check (QPC)#66

Merged
certara-jcraig merged 9 commits intomasterfrom
PMXRD-219-mpc
Jan 11, 2026
Merged

PMXRD-219 Quantitative Predictive Check (QPC)#66
certara-jcraig merged 9 commits intomasterfrom
PMXRD-219-mpc

Conversation

@certara-jcraig
Copy link
Collaborator

@certara-jcraig certara-jcraig commented Jan 3, 2026

Quantitative Predictive Check (QPC) scoring for tidyvpc #68

This PR adds a new Quantitative Predictive Check (QPC)* capability to tidyvpc, providing a numeric scoring framework for continuous VPCs.

Key features

  • New public API: qpcstats() (S3 generic + qpcstats.tidyvpcobj)

    • Computes QPC diagnostics from an existing tidyvpcobj after vpcstats()
    • Produces o$qpc.stats containing:
      • qpc_score (lower is better)
      • component penalties: coverage (median/tails), MAE, drift, sharpness, interval
      • per-quantile wide diagnostics (e.g., coverage, interval score, etc.)
      • supports binless (x) and binning (xbin) continuous workflows
      • supports stratified VPCs, returning per-stratum rows plus an qpc_scope == "overall" row
      • categorical VPCs currently error with a clear message (continuous-only scope)
  • Improved printing

    • Updated print.tidyvpcobj() to display a readable QPCsummary:
      • QPC: qpc_score = ... (lower is better)
      • formatted “Breakdown (0 = best)” for each penalty
      • when stratified, prints a compact qpc_score-by-stratum table

Documentation

  • Added a new vignette: vignettes/tidyvpc_qpc.Rmd

Tests

  • Added tests/testthat/test-qpc.R
    • Coverage for:
      • required precondition (o$stats exists)
      • categorical VPC error behavior
      • continuous workflows across binless/binning, stratification, predcorrect, censoring
      • deterministic adversarial tests (variance inflation and drift)

@certara-jcraig certara-jcraig changed the title PMXRD-219 MPQ PMXRD-219 Quantitative Predictive Check (QPC) Jan 11, 2026
@certara-jcraig certara-jcraig merged commit 45d32e8 into master Jan 11, 2026
4 checks passed
@certara-jcraig certara-jcraig deleted the PMXRD-219-mpc branch January 11, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant