hb-mnl?

[markhwhiteii]

I can't get it to run quickly enough: https://discourse.mc-stan.org/t/troubleshooting-a-maxdiff-hierarchical-multinomial-regression/11235

[andytimm]

I am comfortable speeding up Stan code, but learning about MaxDiff, so was thinking contributing here can be a reasonable way to learn.

Could you confirm you’d still review a PR here if I make it in the next few months? Since it’s been a while since this package been worked on wanted to ask early.

[markhwhiteii]

Yes, definitely would review a PR here. Haven't worked on it recently because I'm gonna have to rework the unit tests that are on codecov, and I've been busy with work/other projects/life. But would definitely love the help, and I'm willing to chat about anything MaxDiff generally.

[andytimm]

Ok, well figured out what was wrong with the model as coded this week.

The main issue was that to_vector(Theta) ~ normal(0, 10); with as many choices as in the example is implausibly wide. Translating that to the logit scale implies a pretty strong belief that 1 brand is likely to get 90%+ of the choice. You can see some good graphs of this phenomenon in Jim Savage's slides here on 18/19: https://rpubs.com/jimsavage/sawtoothcon. Another win for the folk theorem.

Fitting with that knocked down to N(0,.5) takes only about 45 minutes (one chain was much slower than the others, most were done in ~25 - not sure if that is because I was also running other stuff and that core was being hit harder). The resulting model seems pretty sensible, and the values of Theta align with fitting EB to the same data pretty well (EG healthcare is the best message).

I'll do a bit more digging and see if I can't improve on this more, but it does seem like this was the main problem. Since the appropriate sigma in the prior depends on the number of choices, I'll probably try and implement something like the priors here next: https://journals.sagepub.com/doi/abs/10.1509/jmkr.46.6.832_JMR6J?journalCode=mrja&utm_source=pocket_saves.

I'm thinking I'll probably do an initial PR that just implements the model without covariates first, then add in covariates after you confirm there's nothing in the implementation you want modified to fit the package design ideas. Sound good?

[markhwhiteii]

Yeah, that all sounds incredible! Good finds. Definitely would be ideal to have some sort of code that takes number of choices and chooses a sensible default prior. Thanks.