[For Brown Community] Setting up HSSM on Oscar #440
Replies: 3 comments 5 replies
-
|
Getting this error on Oscar: [igrahek@login009 ~]$ module load intel-one-mkl
Lmod has detected the following error: The following module(s) are unknown: "intel-one-mkl"
Please check the spelling or version number. Also try "module spider ..."
It is also possible your cache file is out-of-date; it may help to try:
$ module --ignore_cache load "intel-one-mkl"
Also make sure that all modulefiles written in TCL start with the string #%Module |
Beta Was this translation helpful? Give feedback.
-
|
Sorry the module should be |
Beta Was this translation helpful? Give feedback.
-
|
Hello, a clarification question about which modules to load when: In the GPU example, |
Beta Was this translation helpful? Give feedback.
-
|
Hi Mike, You are right. This does not affect the GPU backend, which relies on JAX for computation |
Beta Was this translation helpful? Give feedback.
-
|
OK thanks. JW, what's the motivation for loading it here? (& why not MKL instead?) |
Beta Was this translation helpful? Give feedback.
-
|
You got me lol. I actually had to go through edit history to see why |
Beta Was this translation helpful? Give feedback.
-
|
HSSM install is failing due to unavailability of Install commands: Running README example: |
Beta Was this translation helpful? Give feedback.
-
|
Confirming this is an issue with running hssm on Oscar. It's weird because our |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
This is a tutorial on how to get HSSM to work on Oscar, Brown University's HPC cluster. This may or may not work with other HPC environments, depending on the setup.
You can choose to work with HSSM either in a
venvor acondaenvironment. The former option, if linked correctly with the Intel MKL library on Oscar, can lead to a significant performance boost on Oscar.Option 1: Using HSSM in
venv(Faster)In general, we recommend that we work with environment created with
venvon Oscar and install packages withpip. However, the set up for PyMC is slightly more involved because we need to tell PyMC where it can find the BLAS library to accelerate its computation. Fortunately, we only need to do this once, unless the path to the library changes. When that happens, we just need to run this process again.Create a
.pytensorrcfile with path to a BLAS libraryThe BLAS library that seems to work well with
pytensoris the Intel one MKL library. We need to load theintel-oneapi-mklmodule and tellpymchow to link to that library:You will see a few paths printed out separated with colons. Copy the path that ends in
intel64after running the above command. Then, create a.pytensorrcfile under your home directory~, and add the following content:Replace
path_just_copiedwith what you have just copied. Be sure to keep the-Lbefore the path. This will ensure thatpymcfinds the lapack library. You only need to do this once unless the LAPACK module gets changed. The file will look like this:Load modules and create environment
We need the
pythonmodule to create the virtual environment and activate it. You can do this in the login node.module load python intel-oneapi-mkl python -m venv .venv source .venv/bin/activateInstall HSSM
Once the environment is loaded, we can install
hssmwithpip:Or you can install the
devversion directly from github:The installation is complete. Please ensure that there is no warning like this:
WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.whenhssmis imported.Optional: Install JAX with GPU support
If you need JAX with GPU support, please perform the installation in an interactive GPU node:
Once the node is up, run the following commands to set up the virtual environment. Note that you also need to load the
cudaandcudnnmodules before proceeding:Then, install the GPU version of
JAX:Finally, you can choose to install the release or dev versions of HSSM.
Option 2: Using HSSM in a conda environment:
The use of
condais discouraged on Oscar. It might be slow in resolving dependencies. If you have to useconda, we recommend usingmambain place ofcondafor considerable speed-up.With the conda environment, you do NOT need to set up the
.pytensorrcfile. In fact, it's not a good idea to have that file in your home directory.Load modules and create environment
We need the
miniforgemodule to create the virtual environment and activate it. It comes withmambainstalled. You can also use theanacondamodule, but you won't be able to use mamba with it.module load miniforge mamba create -n my_env python=3.11 #3.10 also works mamba activate my_envInstall HSSM
HSSM is directly available on
conda-forgenow, so you can directly install it:If you need the dev version of HSSM, then you need to install
pymcfrom conda first, before installing the dev version from github with pip:Optional: Install JAX with GPU support
If you need JAX with GPU support, please perform the installation in an interactive GPU node:
Once the node is up, run the following commands to set up the conda environment. Note that you also need to load the
cudaandcudnnmodules before proceeding:module load miniforge cuda cudnn mamba create -n my_env python=3.11 #3.10 also works mamba activate my_envThen, install the GPU version of
JAX:Finally, follow the instruction in the previous section to install either the release or the dev version of HSSM.
Beta Was this translation helpful? Give feedback.
All reactions