Skip to content

HANDOFF.md

Latest commit

 

History

History
106 lines (69 loc) · 5.61 KB

HANDOFF.md

File metadata and controls

106 lines (69 loc) · 5.61 KB

HANDOFF.md

Session date: 2026-02-11 (session 2)

Summary

This session fixed a test failure caused by the previous session's effect_size refactor. The tau parameter — which controls the adaptive alpha schedule in compute_adaptive_alphas() — was accidentally dropped from simulate_test_DT()'s signature during that refactor. Restored it, regenerated documentation, bumped the patch version to 0.0.0.9301.

Key Decisions Made

  1. tau was restored, not removed from the test: The test (test_adaptive_power_method.R:41) was correct — tau is a meaningful, independent tuning parameter for alpha_method = "adaptive_power". It controls how aggressively the alpha schedule compensates for power decay at deeper tree levels. It was accidentally dropped from simulate_test_DT() when delta_hat was removed, but unlike delta_hat (which became an internal computation), tau has no internal derivation and must remain user-facing.

  2. Default tau = 0.1: Matches manytestsr::compute_adaptive_alphas()'s own default, so existing code that doesn't pass tau is unaffected.

  3. Patch version bump (9300 → 9301): This is a bug fix, not a feature or breaking change, so a patch-level increment is appropriate.

Files Changed and Why

Modified Files

  • R/simulate_test.R (3 changes):

    • Added tau = 0.1 to simulate_test_DT() function signature (line 97)
    • Added @param tau roxygen documentation describing the parameter's role and default
    • Added tau = tau to the internal manytestsr::compute_adaptive_alphas() call (line 149) so the parameter is actually forwarded

  • DESCRIPTION: Version bumped from 0.0.0.9300 to 0.0.0.9301

  • NEWS.md: Added TreeTestSim 0.0.0.9301 section documenting the bug fix

Generated/Updated by roxygen2

  • man/simulate_test_DT.Rd: Regenerated to include tau parameter documentation

Current Blockers or Open Questions

  1. renv out-of-sync: Still present from previous sessions. Every R command shows "The project is out-of-sync." Does not affect functionality.

  2. CLAUDE.md NOTE: R CMD check still reports the pre-existing NOTE about non-standard top-level files (CLAUDE.md, CLAUDE_CODING.md, HANDOFF.md).

  3. Downstream scripts: Any scripts outside the package (especially in ~/repos/manytests-paper) that call simulate_test_DT() or simulate_many_runs_DT() with the old parameter names (beta_base, effN, adj_effN, delta_hat) will still break. This was flagged in the previous session and remains unaddressed.

  4. CLAUDE.md architecture section outdated: Still references beta_base and adj_effN in the architecture description. Should be updated to reflect effect_size, power_decay, and tau.

Important Context to Preserve

The tau parameter

tau is forwarded to manytestsr::compute_adaptive_alphas() only when alpha_method = "adaptive_power". Smaller tau values produce more generous alpha at deeper tree levels (compensating for power loss). With tau = 1, the schedule is flat — all levels get nominal alpha. The parameter is independent of the effect-size parameterization; it controls alpha allocation, not signal strength.

The Core Math (from previous session, still current)

Cohen's d at leaf → power at any level → Beta shape parameter:

Power at level l = Phi(d * sqrt(N_total / (4 * k^l)) - z_{alpha/2})
Beta shape a    = log(power) / log(alpha)
delta_hat       = effect_size / 2    (for compute_adaptive_alphas)

The clamping in effect_size_to_beta is important: power must stay in (alpha, 1) to produce valid beta parameters in (0, 1).

Parameter Mapping (Old → New, established in previous session)

Old New Notes
beta_base effect_size Different scale; not a simple rename
effN (removed) Was always equal to N_total
adj_effN power_decay Same TRUE/FALSE semantics
delta_hat (removed) Derived internally as effect_size / 2
tau tau Unchanged; was accidentally dropped, now restored

Test Results

  • devtools::test(): 773 passed, 0 failed, 0 warnings, 3 skipped (long-running sims)
  • devtools::check(): 0 errors, 0 warnings, 1 pre-existing note

What's Done vs. What Remains

Done (this session)

  • Diagnosed test failure: tau parameter missing from simulate_test_DT() signature
  • Restored tau = 0.1 parameter to simulate_test_DT()
  • Added roxygen @param tau documentation
  • Forwarded tau to compute_adaptive_alphas() call
  • Regenerated documentation (devtools::document())
  • DESCRIPTION bumped to 0.0.0.9301
  • NEWS.md updated with bug fix entry
  • devtools::test() passes (773/773)
  • devtools::check() passes (0 errors, 0 warnings)

Done (previous session, preserved)

  • effect_size parameterization replacing beta_base
  • effect_size_to_beta() helper function
  • power_decay replacing adj_effN
  • root_power in sim_res output
  • All test files updated for new API
  • test_effect_size_param.R with 11 statistical principle tests

Remains / Future Work

  • Check ~/repos/manytests-paper for scripts that call the old API and update them
  • Consider running the skipped long-running FWER tests (tests.notforCRAN.R) to verify statistical properties with the new parameterization
  • Update CLAUDE.md architecture section to reflect new parameter names (effect_size, power_decay, tau instead of beta_base, adj_effN)
  • Consider whether effect_size = 0.5 is the best default for simulate_many_runs_DT()
  • Changes from both sessions have not been committed to git