From 6550ab3fed63251a7ed03b1f5085362719adfb08 Mon Sep 17 00:00:00 2001 From: wwarriner Date: Wed, 29 May 2024 15:50:02 -0500 Subject: [PATCH 1/4] fromatting and typographical fixes --- docs/cheaha/software/software.md | 2 +- docs/education/case_studies.md | 6 +++--- docs/workflow_solutions/getting_containers.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/cheaha/software/software.md b/docs/cheaha/software/software.md index 1f6b50f44..06b6ec9e7 100644 --- a/docs/cheaha/software/software.md +++ b/docs/cheaha/software/software.md @@ -50,4 +50,4 @@ For more information on usage with examples, see [Anaconda Environments](../../w ## Singularity Containers -Containers are a very useful resource for installing software without needing administrator permission. Please read the full documentation about singularity and containers on our [main Singularity page](../../workflow_solutions/getting_containers.md#containers-on-cheaha). +Containers are a very useful resource for installing software without needing administrator permission. Please read the full documentation about Singularity and containers on our [main Singularity page](../../workflow_solutions/getting_containers.md#containers-on-cheaha). diff --git a/docs/education/case_studies.md b/docs/education/case_studies.md index 7085e49da..31183f554 100644 --- a/docs/education/case_studies.md +++ b/docs/education/case_studies.md @@ -126,7 +126,7 @@ You can execute Parabricks on Cheaha using `amperenodes` partition. Maximum numb module load Singularity/3.5.2-GCC-5.4.0-2.26 module load CUDA/11.6.0 -#Run the "pbrun" executable from the singularity image "parabricks-4.2.0-1.sif", and pass the CUDA lib path to make it accessible within the container +#Run the "pbrun" executable from the Singularity image "parabricks-4.2.0-1.sif", and pass the CUDA lib path to make it accessible within the container singularity run --nv parabricks-4.2.0-1.sif /bin/pbrun fq2bam \ --ref parabricks_sample/Ref/Homo_sapiens_assembly38.fasta \ --in-fq parabricks_sample/Data/sample_1.fq.gz parabricks_sample/Data/sample_2.fq.gz \ @@ -236,7 +236,7 @@ $ module load CUDA/11.6.0 $ nvidia-smi -Fri Nov 3 12:38:24 2023 +Fri Nov 3 12:38:24 2023 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.86.10 Driver Version: 535.86.10 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ @@ -252,7 +252,7 @@ Fri Nov 3 12:38:24 2023 | N/A 32C P0 112W / 300W | 16631MiB / 81920MiB | 96% Default | | | | Disabled | +-----------------------------------------+----------------------+----------------------+ - + +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | diff --git a/docs/workflow_solutions/getting_containers.md b/docs/workflow_solutions/getting_containers.md index 928ccbb3b..d45bde290 100644 --- a/docs/workflow_solutions/getting_containers.md +++ b/docs/workflow_solutions/getting_containers.md @@ -51,12 +51,12 @@ It's important to note that both `run` and `exec` enter the container as part of !!! important - `singularity shell` is not executable via shell scripts. Any singularity commands in a batch script should be `run` or `exec` instead. + `singularity shell` is not executable via shell scripts. Any Singularity commands in a batch script should be `run` or `exec` instead. ### Singularity Paths -By default, Singularity containers have limited access to the general filesystem. Containers get default access to the `/home` directory as well as the directory the container was run from. If you run the container from `$HOME` but try to access files in `$USER_DATA`, you will see an error. In order to give a container access to other directories, use the `-B` or `--bind` option when invoking the container. For instance, if I wanted to use `run` on a container that had an input option called `-i` and give the container access to a subfolder called `my_data` in a project space called `UABRC`, the singularity command would look like: +By default, Singularity containers have limited access to the general filesystem. Containers get default access to the `/home` directory as well as the directory the container was run from. If you run the container from `$HOME` but try to access files in `$USER_DATA`, you will see an error. In order to give a container access to other directories, use the `-B` or `--bind` option when invoking the container. For instance, if I wanted to use `run` on a container that had an input option called `-i` and give the container access to a subfolder called `my_data` in a project space called `UABRC`, the Singularity command would look like: ``` bash singularity run --bind /data/project/UABRC/my_data image.sif -i /data/project/UABRC/my_data From 3e9fe41b750cb3d97d7fe4460f62fabf6066c343 Mon Sep 17 00:00:00 2001 From: wwarriner Date: Wed, 29 May 2024 15:50:33 -0500 Subject: [PATCH 2/4] remove references to module system, explain that the command is directly available on nodes --- docs/education/case_studies.md | 10 ++-------- docs/workflow_solutions/getting_containers.md | 8 +------- 2 files changed, 3 insertions(+), 15 deletions(-) diff --git a/docs/education/case_studies.md b/docs/education/case_studies.md index 31183f554..257813a01 100644 --- a/docs/education/case_studies.md +++ b/docs/education/case_studies.md @@ -42,13 +42,7 @@ Parbaricks 4.x are available as containers in the [NGC Catalog](https://catalog. Parabricks 4.x container image can be installed on Cheaha using a Singularity container. More details on usage of Singularity container on Cheaha can be found in the [Containers Page](../workflow_solutions/getting_containers.md). -To install Parabricks using Singulairty, load the `Singularity 3.x` module from Cheaha as, - -```bash -module load Singularity/3.5.2-GCC-5.4.0-2.26 -``` - -Go to the NGC catalog page and copy the image path to pull the desired containers of Parabricks using Singularity. Here, the generic container is pulled using Singularity. The image path is in “nvcr.io/nvidia/clara/clara-parabricks" and the tag is 4.2.0-1. The container image name `parabricks-4.2.0-1.sif` is an user-derived name. +Singularity is installed directly on all nodes, in accordance with the latest security recommendations. To install Parabricks using Singularity, there is no need to load a module. Go to the NGC catalog page and copy the image path to pull the desired containers of Parabricks using Singularity. Here, the generic container is pulled using Singularity. The image path is in “nvcr.io/nvidia/clara/clara-parabricks" and the tag is 4.2.0-1. The container image name `parabricks-4.2.0-1.sif` is an user-derived name. ![!Parabricks container.](./images/parabricks_container.png) @@ -56,7 +50,7 @@ Go to the NGC catalog page and copy the image path to pull the desired container singularity pull parabricks-4.2.0-1.sif docker://nvcr.io/nvidia/clara/clara-parabricks:4.2.0-1 ``` -After the image `parabricks-4.2.0-1.sif` is successfully created, you can run singularity image `parabricks-4.2.0-1.sif` with all input and output parameters. Various ways of running singularity image can be found in the [Containers Page](../workflow_solutions/getting_containers.md). +After the image `parabricks-4.2.0-1.sif` is successfully created, you can run Singularity image `parabricks-4.2.0-1.sif` with all input and output parameters. Various ways of running Singularity image can be found in the [Containers Page](../workflow_solutions/getting_containers.md). Running `singularity shell` helps to navigate through the containers directory to verify the path of the software executable and later use the path outside the container to run the software. Following are the commands to run the container using `singularity shell` and traverse through the directories inside the contianer. diff --git a/docs/workflow_solutions/getting_containers.md b/docs/workflow_solutions/getting_containers.md index d45bde290..84098c41a 100644 --- a/docs/workflow_solutions/getting_containers.md +++ b/docs/workflow_solutions/getting_containers.md @@ -14,13 +14,7 @@ It is often a good idea to search the Github repo for an application or pipeline ## Containers on Cheaha -Using containers on Cheaha bypasses the need to message support to install necessary software. Containers can be downloaded by any user into their personal space and used immediately without admin permission. as mentioned above, you will need to use Singularity containers on Cheaha. You can find all of the Singularity modules using the following command: - -``` bash -module spider Singularity -``` - -It's highly recommended to only use Singularity versions 3+. +Using containers on Cheaha bypasses the need to message support to install necessary software. Containers can be downloaded by any user into their personal space and used immediately without admin permission. As mentioned above, you will need to use Singularity containers on Cheaha. In accordance with the most recent security recommendations, Singularity is installed directly on each node, so module commands are not needed to use Singularity. ### Pull Singularity Images From 9b330b492d38c653742d87d4052161dbca7e14ba Mon Sep 17 00:00:00 2001 From: wwarriner Date: Wed, 29 May 2024 15:54:09 -0500 Subject: [PATCH 3/4] add note about Singularity change on Cheaha software page --- docs/cheaha/software/software.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/cheaha/software/software.md b/docs/cheaha/software/software.md index 06b6ec9e7..87721f4c1 100644 --- a/docs/cheaha/software/software.md +++ b/docs/cheaha/software/software.md @@ -51,3 +51,9 @@ For more information on usage with examples, see [Anaconda Environments](../../w ## Singularity Containers Containers are a very useful resource for installing software without needing administrator permission. Please read the full documentation about Singularity and containers on our [main Singularity page](../../workflow_solutions/getting_containers.md#containers-on-cheaha). + + +!!! note + +Singularity is not installed via the [module system](./modules.md), in accordance with the most recent security recommendations. The `singularity` command may be used directly on any compute node, without the need for `module load`. + From 92dbd0246b4810d8f3c6571682069aa360c8a8d2 Mon Sep 17 00:00:00 2001 From: wwarriner Date: Wed, 29 May 2024 15:54:27 -0500 Subject: [PATCH 4/4] formatting --- docs/cheaha/software/modules.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/cheaha/software/modules.md b/docs/cheaha/software/modules.md index 26544f64a..dd515e0a8 100644 --- a/docs/cheaha/software/modules.md +++ b/docs/cheaha/software/modules.md @@ -145,6 +145,7 @@ When using modules in Cheaha, we recommend users to follow these best practices 2. Be selective and only load a specific module version that you need for your current workflow. Loading unnecessary modules can lead to conflicts and inefficiencies. 3. Before loading modules in a shell/bash/sbatch script, use a clean shell by using `module reset` at the beginning. + - What it does: - Clearing loaded modules. - Loading default modules specified by the system administrator. @@ -153,7 +154,7 @@ When using modules in Cheaha, we recommend users to follow these best practices - Why it is a best-practice: - Ensures reproducibility by starting with a clean environment. - Manages software dependencies effectively. - + Using `module reset` before loading modules separates what software is loaded in the working shell from the software loaded in the script shell. Be aware that forked processes (like scripts) and Slurm commands inherit the environment variables of the working shell, including loaded modules. Here is an example that shows module conflict between cuda11.8 and cuda11.4 versions that may lead to unexpected behavior or an erroneous output. ```bash