-
Notifications
You must be signed in to change notification settings - Fork 3
feat: Add the checkbig convergence criteria #207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds the CHECKBIG convergence metric from Fortran NPAGFULLA, which measures parameter stability by calculating the median of relative parameter changes across support points. While the metric is calculated and logged, the convergence criterion continues to use the standard f1-f0 difference approach.
Key changes:
- Implemented
calculate_checkbig()function to compute median relative parameter changes - Added
theta_oldfield to track previous iteration's theta values - Enhanced convergence logging to display both f1-f0 and CHECKBIG metrics for diagnostics
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
| Branch | checkbig |
| Testbed | rust-loki |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result seconds (s) (Result Δ%) | Upper Boundary seconds (s) (Limit %) |
|---|---|---|---|
| bimodal_ke_npag | 📈 view plot 🚷 view threshold | 44.12 s(+20.20%)Baseline: 36.71 s | 47.87 s (92.16%) |
| bimodal_ke_npod | 📈 view plot 🚷 view threshold | 60.77 s(+24.42%)Baseline: 48.84 s | 68.68 s (88.48%) |
| bimodal_ke_postprob | 📈 view plot 🚷 view threshold | 20.68 s(+22.86%)Baseline: 16.83 s | 23.25 s (88.97%) |
|
| Branch | checkbig |
| Testbed | rust-moan |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result seconds (s) (Result Δ%) | Upper Boundary seconds (s) (Limit %) |
|---|---|---|---|
| bimodal_ke_npag | 📈 view plot 🚷 view threshold | 18.75 s(-23.53%)Baseline: 24.51 s | 30.08 s (62.31%) |
| bimodal_ke_npod | 📈 view plot 🚷 view threshold | 10.41 s(-0.49%)Baseline: 10.47 s | 10.70 s (97.37%) |
| bimodal_ke_postprob | 📈 view plot 🚷 view threshold | 3.46 s(-0.18%)Baseline: 3.47 s | 3.54 s (97.73%) |
|
The tests are failing because the medians are stabilized long before the likelihood converges. I think we really need to think this criteria through. |
No description provided.