This repository contains the work I did during my third year of ENSTA engineering curriculum at ONERA under the supervision of two researchers, Mathieu Balesdent and Loïc Brevault, in the Signal Processing Department.
In the context of space engineering, the first part of this project was about getting familiar with a sensitivity indicator adapted to multidisciplinary optimisation, the Hilbert-Schmidt Independence Criterion(HSIC). This indicator is based on the Kernel Methods machinery, which makes it all the more interesting to study it in the machine learning era. Kernel methods (see my repo) are very interesting tools in general.
From the theoretical perspective, you can think to the Reproducing Kernel Hilbert Spaces (RKHS) and embedding of probability distributions into functional space; or from the experimental perspective with the generalization of a whole set of machine learning algorithms to non-linear spaces. There is nothing magic though, kernel methods can be non performant in high dimensional spaces. During my time at ONERA, I was introduced to these methods for the first time, through the use of kernel methods to make a sensitivity analysis. Sensitivity analysis is a subfield of uncertainty quantification, that is the estimate and modeling, whether with frequentist or bayesian approach, of the uncertainty in our model. Sensitivity analysis is concerned with the following question: "Is our output equally sensitive to all of its entries ?" The question might not make sense at first, you could be tempted to say: "I guess it depends 'where' you are in you domain of definition", and you would be right. A function f of x and y could be very sensitive to x and not at all to y in a region, but the opposite in another region. The more natural way to measure the sensitivity is to compute the partial derivatives, but it is often costly (two evaluations of functions) and possibly imprecise, strongly depending on the local regularity of your function. In the worst, but common, case where the function is non-differentiable, you can not rely on such an estimator. Finally, while the partial derivative is quite informative, it is only a ponctual information.
The method I employed here is based on the HSIC estimators, through the embedding of probability distributions into functional spaces (a RKHS). As often, in a modern approach, one rather relies on a probabilistic point of view (by nature non local) to model the input-output relationship of our system. By giving to your random vector of entries X (possibly independant) a joint probabilistic law, you get an output random vector Y = f(X). The objective is to quantify the independance between an chosen entry and the output Y. The general idea is similar to dimension reduction, you want to remove the features that, statistically, have very little impact on your prediction/model. In the context of space engineering, you are typically in the setting where multidisciplinary fields influence each other. Simplifying the problem is essential to make the computation feasible. Multidisciplinary optimization brings a partial answer to this issue, sensitivity analysis brings another.
This second part remains in the continuity of the first one, but explores a different topic: the non-parametric bayesian estimation of probability distribution function. I implemented the LAMDA method, and in particular replicated the results of this paper. The objective is to ensure multidisciplinary feasibility via a Monte-Carlo method, rather than a brute force resolution of a complicated system of equation via a fixed point algorithms. This part implements the FORM method, coupled with a Bayesian algorithm to infer a posterior over the output random variable.
OpenTurns, a Python librairy dedicated to the computation of statistical quantities, is heavily utilized here.