(LDA): sparse initialization rather than uniformly random initialization

[hucheng]

The phenomenon of LDA training is that the first several training is very costly, this is largely due to the uniformly random initialization that the word-topic thus doc-topic is quite dense.

There are two approaches：

1. sparse initialization that constraints a word to only a part (like 1%) (randomly) of all topics, and for each tokens of that word, randomly sample from those constrained topics rather than all topics.
2. First use part of corpus (like 1%) to train several iterations to initialize the word-topic distribution, which should be quite sparse than uniformly random initialization.

[bhoppi]

We can split the perplexity into two parts, e.g. word perplexity and doc perplexity. Then we will look into the impact of the different initialization strategies to the two perplexity parts.

[hucheng]

Great. Look forward to the experimental result of both word perplexity and doc perplexity.