sparse dirichlet discrete

[stephentu]

I think Fritz and I had a conversation somewhat related to this topic, but I sort of forgot the outcome. If I have a DD where the dim is very large, and I expect the non-zero entries of the suffstats (e.g. the counts) to be very sparse, what's the right way to do this in distributions?

Essentially I want a DD where the counts[] is a Sparse<> instead of float[]. Would it be worth created a separate model which is SparseDD?

[fritzo]

The DPD datatype degenerates to DD when shared.beta0 = 0. In this case, you'll get a dense shared.betas analogous to DD shared.alphas, and you'll get a sparse group.counts. Would that work for you?

[stephentu]

Yes this will be sufficient for now. Thanks!

[fritzo]

A slight correction: the DPD with shared.beta0=0 is equivalent to the DD via the equation

dpd.shared.betas[i] * dpd.shared.alpha = dd.shared.alphas[i]