Speed up null fitting for categorical X's via Fisher scoring + line search

[adw96]

The goal here is not to have a general faster fitting but some major speed ups for the common case of categorical covariates and pseudoHuber median g(.).

It turns out that #90 is impossible (no closed form for pH median g(.)), but we can speed things up.

I'm seeing perhaps 1000x faster, basically instantaneous fitting, but I haven't broadly tested it over a range of p and J. (n doesn't matter here)

In the one case I investigated where estimates differ from fit_null, this approach had higher likelihood and it strictly enforces the g constraints (ie constraint_tol is trivially 0).

What's it actually doing? Fisher scoring (in the free parameters) + a line search to ensure the likelihood is increasing at each step.

Lots of ChatGPT here, which is fine because I know what I want.

Not ready for review, but opening to give some updates to @svteichman

Next steps (AW) - taken from my notes in fit_null_discrete

  ## Currently this works for 
  #### two groups with X's (1, 0) and (1, 1)
  #### testing the first column, last column also constrained
  
  #### it is equivalent to fit_null with k_constr=2, j_constr=1, j_ref=J,
  
  ## It needs to be generalized to consider 
  #### multiple categories (can we assume identity-like then back transform? is this constraint and likelihood-preserving?) 
  #### different columns for testing
  #### the same inputs as fit_null and comparable convergence statistics

[adw96]

• preliminary generalisation to ncol(X) == nrow(distinct_xx) case for general ncol(X)
• preliminary confirmation that it's doing the right thing
• more extensive testing that it's doing the right thing
• more extensive testing that it's likelihood is no worse (or not much worse)
• align IO with current
• generalise to generic j_ref
• testing for correctness
• documentation of inputs
• integrate into codebase

[adw96]

Hi @svteichman -- I think this is ready for your feedback and edits. A "wish list", not all essential

• integration into codebase
• documentation, inc w holes for me to fill
• matching formats of output of original null fitting
• other things I'm too tired to think of now...

Please let me know your questions -- and thank you!!!!

[svteichman]

Very exciting! I'll take a look and get started on integrating this into the codebase.

[svteichman]

This PR is superseded by PR #179, and can be closed once that is reviewed and merged (although we may want to remember someday that this PR is the place where Amy derived the second derivative of the pseudohuber function!!)