One task I've seen people work on is the ability to quantify the type of uncertainty in a dataset/model. How much of the predictive uncertainty is irreducible (aleatoric) vs epistemic (limited data)? I think we can use Hypernetworks + MSE/NLL to quantify this difference:
epistemic_noise = HNET+MSE
aleatoric_noise + epistemic_noise = HNET+NLL
aleatoric_noise = (HNET+NLL) - (HNET + MSE)
This could have utility in data acquisition tasks like bayesian optimization, reinforcement learning, etc.
One task I've seen people work on is the ability to quantify the type of uncertainty in a dataset/model. How much of the predictive uncertainty is irreducible (aleatoric) vs epistemic (limited data)? I think we can use Hypernetworks + MSE/NLL to quantify this difference:
epistemic_noise = HNET+MSE
aleatoric_noise + epistemic_noise = HNET+NLL
aleatoric_noise = (HNET+NLL) - (HNET + MSE)
This could have utility in data acquisition tasks like bayesian optimization, reinforcement learning, etc.