Independence and marginals in `DiscreteDistribution`

[Mv77]

This is not an issue or bug, but rather a desired feature that I think would make our lives easier and more elegant.

It would be useful if multivariate discrete distributions had:

1. A property telling us whether its dimensions are independent.
2. A method to extract joint distributions of subsets of its dimensions. E.g., if I have the joint distribution of permanent shocks to income, transitory shocks to income, and risky returns ShkDstn, it would be useful to be able to do something like InkShkDstn = ShkDstn.marginal(dims=[0,1]).
3. A method to determine whether its dimensions are independent. Since we are dealing with discrete distributions, this should be easy.

What is the value of all this?
1. Redundancy reduction: IndShockConsumerType currently tracks IncShkDstn, TranShkDstn, and PermShkDstn. It'd be good if we kept track of a single object.
2. In-code determination of the methods/speedups that can be used: Harmenberg's method works only if permanent shocks are independent from all other shocks (I think); a portfolio problem can be solved much more quickly if the return factor is independent of other shocks. Currently we ask the user to tell us whether these things are true with arguments like IndepDstnBool. If would be good if the user could simply give us a joint distribution of shocks and we figured out the rest.
3. The above point would help to reduce hard-coded features and make things more modular: IndShockConsumerType knows its shocks are independent because we have hard-coded its income process. If we want to move towards a world where we can pass an income distribution constructed elsewhere, we have to be able to determine the properties of that distribution, for the reasons above.

[llorracc]

@Mv77,

This is a perfect example of the reason I've been advocating for a long time that we should simply adopt one of the sophisticated external architectures that already exist for handling distributions. All of them have built in ways for doing these and all the other kinds of sophisticated things one might want to do with distributions. In my mind, the two leading candidates are TensorFlow and sympy though pytorch might be an option as well, and stan seems to have a big following.

I've asked @nicksawhney to familiarize himself with the way Mathematica does these things (partly because there are very good tutorials, and because as best I can tell, sympy and TensorFlow implement much the same set of ideas in slightly different ways.

One crosscutting point that would apply whatever method we adopt: If we want to allow our shocks to be correlated, then at the simulation stage it will probably be best to draw all the shocks simultaneously at the beginning of the 'period' in which they are all realized. And probably for the solution procedure it would make sense to define the abstract mathematical description of the shocks at the same point. The only argument against this that I can think of is that we might be solving a problem in which some of the shocks are contingent on conditions that may not matter. Like, if the person is not employed, then it would be a waste of computational resources to draw the shocks for the covariance of their income-if-employed with the rate of return. Still, the generality and simplicity of drawing all the shocks at once seems compelling -- especially since that's what Pablo does in Dolo and whenever we don't have a strong reason to do things differently from Dolo I want to adopt Pablo's choices.

One other subtlety is the desirability of distinguishing between draws of aggregate shocks (like the rate of return) and idiosyncratic shocks (like employment income). This probably militate for specifying the aggregate shock process as an input to the individual's problem, and then specifying the individual's shocks as marginals contingent on the aggregate shocks.

@nicksawhney, @sbenthall, @alanlujan91

[sbenthall]

@Mv77 I agree that this would be good functionality.

@llorracc I agree that the best way towards this is to see if we can get it from an imported library.

As we've discussed, I think the practical blocker to experimenting with alternative Distribution implementations is #1105. Once that is done, it should open up a lot of possibilities.

I've made been using this issue label for tickets related to the Distribution architecture.

Once #1105 is completed, I would recommend systematically:

• listing all the features we use and anticipate using for distributions.
• going through the various libraries and checking off, for each one, which of those desired features it supports

[sbenthall]

I've begun a wiki page to address this and other issues about possible use of external libraries for probability distributions.

https://github.com/econ-ark/HARK/wiki/Distribution-Libraries

I'd like us to use it to:

• Python libraries that have an existing representation of probability distributions
• Features we want in HARK for our distribution objects

For now, we can build these lists out separately, and should not let either list bias the creation of the other.

Later, we can survey across the two lists and see which libraries get us the farthest to our goals.