NUQ Uncertainty

This repository implements an uncertainty estimation kernel method based on Nadaraya-Watson kernel regression (see NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks by N. Kotelevskii et al.)

Environment

Important: we recommend having Jupyter Lab installed in the base conda environment. For the best experience, you may also install nb_conda_kernels and ipywidgets.

1. Create conda environment:

   $ conda env create --file environment.yaml

2. Activate it:

   $ conda activate ray-env

How to use

See classification and regression examples for more details.

Classification

from nuq import NuqClassifier

nuq = NuqClassifier()
nuq.fit(X_train, y_train)
preds, log_uncs = nuq.predict(X_test, return_uncertainty="epistemic")

Regression

from nuq import NuqRegressor

nuq = NuqRegressor()
nuq.fit(X_train, y_train)
preds, log_uncs = nuq.predict(X_test, return_uncertainty="epistemic")

Installation

(temporary local only) Pip doesn't handle .toml files correctly, thus using setup.py:

python setup.py develop

To uninstall:

pip uninstall nuq

Development

1. Install pre-commit (config provided in this repo)

   $ pre-commit install

2. (optional) Run against all the files to check the consistency

   $ pre-commit run --all-files

3. You may also run black and isort to keep the files style-compliant

   $ isort .; black .

4. Proposed linter is flake8

   $ flake8 .

5. One can run tests via pytest

   $ pytest