Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Contributions.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,4 +52,4 @@ Keep examples organized in respective per example directories following the esta

## Reference Materials

- For a nice overview of GitHub's markdown syntax [see here](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax).
- CCR uses[GitHub's markdown syntax](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax), a version of markdown that contains some extensions such as tables and nesting block content inside list elements.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ This repository uses placeholders denoted by square brackets for values that nee
|-------------------------|-------------------------------------------|
| `[cluster]` | ub-hpc, faculty |
| `[partition]` | general-compute, debug, industry, scavenger, ub-laser, [other available options](https://docs.ccr.buffalo.edu/en/latest/hpc/clusters/#ub-hpc-compute-cluster) |
| `[qos]` | Typically the same as `[partition]` - refer to [CCR docs](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#slurm-directives-partitions-qos) for more information. |
| `[qos]` | Typically the same as `[partition]` - refer to [CCR documentation](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#slurm-directives-partitions-qos) for more information. |
| `[SlurmAccountName]` | Use the `slimits` command to see what accounts you have access to. If not specified, your default account will be used. |
| `[CCRUsername]` | Your CCR username |
| `[YourGroupName]` | Your project's group name (often PI username or company name) |
Expand Down
2 changes: 1 addition & 1 deletion containers/2_ApplicationSpecific/AlphaFold-3/EXAMPLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Change to your AlphaFold-3 directory
cd /projects/academic/[YourGroupName]/AlphaFold-3
```

Then create the sample `fold_input.json` from [here](https://github.com/google-deepmind/alphafold3?tab=readme-ov-file#installation-and-running-your-first-prediction)
Then create the sample `fold_input.json` from [the github example](https://github.com/google-deepmind/alphafold3?tab=readme-ov-file#installation-and-running-your-first-prediction)

```
mkdir -p ./af_input ./af_input_inference ./af_output
Expand Down
6 changes: 3 additions & 3 deletions containers/2_ApplicationSpecific/AlphaFold-3/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> [!NOTE]
> AlphaFold 3 requires a model parameters file.
> You must request the file with [this form](https://forms.gle/svvpY4u2jsHEwWYS6) and agree to the software [terms of use](https://github.com/google-deepmind/alphafold3/blob/main/WEIGHTS_TERMS_OF_USE.md)
> You must request the file by completing the [AlphaFold 3 model parameter request form](https://forms.gle/svvpY4u2jsHEwWYS6) and agree to the software [terms of use](https://github.com/google-deepmind/alphafold3/blob/main/WEIGHTS_TERMS_OF_USE.md)
> If you are granted access to the file, Google will email you a download link.

Here is an example to setup the parameters file for use:
Expand Down Expand Up @@ -40,7 +40,7 @@ Please refer to CCR's [container documentation](https://docs.ccr.buffalo.edu/en/
> Apptainer is not available on the CCR login nodes and the compile nodes may not provide enough resources for you to build a container. We recommend requesting an interactive job on a compute node to conduct this build process.

A GPU is NOT needed to build the AlphaFold container.
See CCR docs for more info on [running jobs](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission)
See CCR documentation for more info on [running jobs](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission)

```
salloc --cluster=ub-hpc --partition=debug --qos=debug --mem=0 --exclusive --time=01:00:00
Expand Down Expand Up @@ -178,7 +178,7 @@ Abridged expected output:
> [...]
> ```

See the [EXAMPLES file](./EXAMPLES.md) for more info.
See the [EXAMPLES file](./EXAMPLES.md) for more information.

## Sample Slurm scripts

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

NOTE:
ALphaFold 3 requires a model parameters file.
You must request the file with [this form](https://forms.gle/svvpY4u2jsHEwWYS6) and agree to the software [terms of use](https://github.com/google-deepmind/alphafold3/blob/main/WEIGHTS_TERMS_OF_USE.md)
You must request the file by completing the [AlphaFold 3 model parameter request form](https://forms.gle/svvpY4u2jsHEwWYS6) and agree to the software [terms of use](https://github.com/google-deepmind/alphafold3/blob/main/WEIGHTS_TERMS_OF_USE.md)

If you are granted access to the file, Google will email you a download link.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

To build and use the PyRosetta version you must have a PyRosetta license.
You must either complete the application for a non commercial license or purchase
a commercial license - see [PyRosetta Licensing](https://els2.comotion.uw.edu/product/pyrosetta) for more info.
a commercial license - see the [PyRosetta Licensing](https://els2.comotion.uw.edu/product/pyrosetta) for more information.

## Building the container

Expand Down
6 changes: 3 additions & 3 deletions containers/2_ApplicationSpecific/BindCraft/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ NOTE: for building on the ARM64 platform see [BUILD-ARM64.md](./BUILD-ARM64.md)

Apptainer is not available on the CCR login nodes and the compile nodes may not provide enough resources for you to build a container. We recommend requesting an interactive job on a compute node to conduct this build process.<br/>
Note: a GPU is NOT needed to build the BindCraft container<br/>
See CCR docs for more info on [running jobs](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission)
See CCR documentation for more info on [running jobs](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission)

```
salloc --cluster=ub-hpc --partition=debug --qos=debug --account="[SlurmAccountName]" \
Expand Down Expand Up @@ -43,7 +43,7 @@ There are two sample .def files, [FreeBindCraft.def](./FreeBindCraft.def) and [F
The former does not install PyRosetta, the latter does.
To build and use the PyRosetta version you must have a PyRosetta license.
You must either complete the application for a non commercial license or purchase
a commercial license - see [PyRosetta Licensing](https://els2.comotion.uw.edu/product/pyrosetta) for more info.
a commercial license - see the [PyRosetta Licensing](https://els2.comotion.uw.edu/product/pyrosetta) for more information.

The following example uses the non PyRosetta version.
To build with PyRosetta see [Build with PyRosetta](./BUILD_with_PyRosetta.md)
Expand Down Expand Up @@ -192,7 +192,7 @@ Sample output:
> --interactive Force interactive mode to collect target settings and options
> ```

See the [EXAMPLE file](./EXAMPLES.md) for more info.
See the [EXAMPLE file](./EXAMPLES.md) for more information.

## Sample Slurm scripts

Expand Down
2 changes: 1 addition & 1 deletion containers/2_ApplicationSpecific/OpenFOAM/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Please refer to CCR's [container documentation](https://docs.ccr.buffalo.edu/en/
1. Start an interactive job

> [!IMPORTANT]
> Apptainer is not available on the CCR login nodes and the compile nodes may not provide enough resources for you to build a container. We recommend requesting an interactive job on a compute node to conduct this build process.<br/> See CCR docs for more info on [running jobs](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission)
> Apptainer is not available on the CCR login nodes and the compile nodes may not provide enough resources for you to build a container. We recommend requesting an interactive job on a compute node to conduct this build process.<br/> See CCR documentation for more information on [running jobs](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission)

```
salloc --cluster=ub-hpc --partition=debug --qos=debug --mem=64GB --time=01:00:00
Expand Down
8 changes: 4 additions & 4 deletions containers/2_ApplicationSpecific/fds/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Example FDS container

The Fire Dynamics Simulator (FDS) is a large-eddy simulation (LES) code for low-speed flows, with an emphasis on smoke and heat transport from fires. For more information on this program, visit the [website](https://pages.nist.gov/fds-smv/).
The Fire Dynamics Simulator (FDS) is a large-eddy simulation (LES) code for low-speed flows, with an emphasis on smoke and heat transport from fires. For more information on this program, visit the [NIST Fire Dynamics Simulator website](https://pages.nist.gov/fds-smv/).

## Pulling the container

Expand All @@ -21,15 +21,15 @@ After the pull completes, the Apptainer image will be saved as `fds_6.7.9.sif` i
> This container includes Intel MPI Library 2021.6. CCR's `ccrsoft/2023.01` [software environment](https://docs.ccr.buffalo.edu/en/latest/software/releases/#202301) includes a module for `intel/2022.00` which includes `impi 2021.5.0` These two versions of MPI are alike enough that
> this container works when loading the `intel/2022.00` module. However, the user should be aware with future versions of FDS and CCR's intel modules, that you must align MPI versions on the system with the container in order for them to work properly.
>
> See [here](https://docs.ccr.buffalo.edu/en/latest/howto/containerization/#building-mpi-enabled-images) for more info.
> See [CCR's documentation on building MPI enabled images](https://docs.ccr.buffalo.edu/en/latest/howto/containerization/#building-mpi-enabled-images) for more information.

## Running the container image

You can run FDS either in an interactive job or non-interactively by using a bash script (recommended).

The following information on running FDS is provided as an example only. The [`radiator.fds`](radiator.fds) input file is taken from the FDS website and used as an example. This will run on, at most, 3 tasks. Your real world work may scale to more tasks.

FDS is capable of running across multiple nodes but this should only be utilized if your problem requires more than the number of CPUs in a compute node. Running FDS across multiple nodes will increase the time it takes for your problem to compute. The specifications for available compute nodes in CCR's UB-HPC cluster can be found [here](https://docs.ccr.buffalo.edu/en/latest/hpc/clusters/#ub-hpc-detailed-hardware-specifications).
FDS is capable of running across multiple nodes but this should only be utilized if your problem requires more than the number of CPUs in a compute node. Running FDS across multiple nodes will increase the time it takes for your problem to compute. The specifications for available compute nodes in CCR's UB-HPC cluster can be found in the [Hardware Specification section of CCR's documentation](https://docs.ccr.buffalo.edu/en/latest/hpc/clusters/#ub-hpc-detailed-hardware-specifications).

Please also refer to the [FDS documentation](https://pages.nist.gov/fds-smv/manuals.html) to properly setup your requests for CPUs and tasks.

Expand All @@ -53,7 +53,7 @@ export I_MPI_PMI_LIBRARY=/opt/software/slurm/lib64/libpmi.so

Navigate to your build directory. Ensure the [`radiator.fds`](radiator.fds) file is present. This input file is taken from the FDS website and will be used for the example.

Request an interactive job with `salloc` to gain access to a compute node, as documented [here](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission). Once the compute node is ready, use `srun` to run FDS on the allocated node:
Request an interactive job with `salloc` to gain access to a compute node, as per [CCR's interactive job documentation](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission). Once the compute node is ready, use `srun` to run FDS on the allocated node:

```
srun --jobid=[JobID] mpirun -np 3 apptainer exec fds_6.7.9.sif fds radiator.fds
Expand Down
2 changes: 1 addition & 1 deletion containers/2_ApplicationSpecific/seurat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ A brief guide to building the seurat container for scRNA analysis follows:
1. Start an interactive job

Apptainer is not available on the CCR login nodes and the compile nodes may not provide enough resources for you to build a container. We recommend requesting an interactive job on a compute node to conduct this build process.
See CCR docs for more info on [submitting an interactive job](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission).
See CCR documentation for more info on [submitting an interactive job](https://docs.ccr.buffalo.edu/en/latest/hpc/jobs/#interactive-job-submission).

```
CCRusername@cpn-h23-04:~$ salloc --cluster=ub-hpc --partition=general-compute --qos=general-compute --mem=32GB --time=02:00:00
Expand Down
2 changes: 1 addition & 1 deletion containers/2_ApplicationSpecific/vasp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Refer to CCR's [container documentation](https://docs.ccr.buffalo.edu/en/latest/

## Running container

1. Download [O atom](https://www.vasp.at/wiki/index.php/O_atom) VASP example
1. Download the [O atom](https://www.vasp.at/wiki/index.php/O_atom) VASP example
for performing a standard calculation for a single oxygen atom in a box:

```
Expand Down
8 changes: 4 additions & 4 deletions containers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ This directory contains example workflows for running containers on CCR's cluste

At CCR, we use Singularity/Apptainer which allows you to execute software without requiring root privilages. CCR provides some commonly used container images, but you can also build your own (from sources like DockerHub, Podman, etc.) and run them with Apptainer. CCR provides a few pre-built containers available in: `/util/software/containers`. For in-depth information about Apptainer, you can refer to the official [Apptainer documentation](https://apptainer.org/documentation/).

## Getting Started ([0_Introductory/](./0_Introductory/README.md))
## Getting Started ([Introductory/](./0_Introductory/README.md))

Here we present a minimalist example of container usage in CCR's HPC environment. Python is used as an example due to it being a common software that benefits from containerization.

## Advanced Topics ([1_Advanced/](./1_Advanced/README.md))
## Advanced Topics ([Advanced/](./1_Advanced/README.md))

Coming Soon

## Application Specific Containers ([2_ApplicationSpecific/](./2_ApplicationSpecific/README.md))
## Application Specific Containers ([ApplicationSpecific/](./2_ApplicationSpecific/README.md))

This directory provides container workflows tailored for specific applications (e.g., Conda, Abaqus, Juicer). Not every software package has a container example here, but a [table](./2_ApplicationSpecific/README.md#table-of-topics) of available workflows is provided for easier navigation.
This directory provides container workflows tailored for specific applications (e.g., Conda, Abaqus, Juicer). Not every software package has a container example here, but a [table of available workflows](./2_ApplicationSpecific/README.md#table-of-topics) is provided for easier navigation.
4 changes: 2 additions & 2 deletions easybuild/0_Introductory/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Here we update the name of the EB recipe from `SAMtools-1.18-GCC-12.3.0.eb` to `
cp /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/Core/easybuild/4.9.4/easybuild/easyconfigs/s/SAMtools/SAMtools-1.18-GCC-12.3.0.eb SAMtools-1.18-GCC-11.2.0.eb
```

The unmodified version of the file is available in this directory [here](./SAMtools-1.18-GCC-12.3.0.eb).
The [unmodified version of the SAMtools EB recipe file](./SAMtools-1.18-GCC-12.3.0.eb) can be viewed for reference.

3. Open the recipe file in your favorite editor. The first thing to change is the toolchain line - change from 12.3.0 to 11.2.0:

Expand Down Expand Up @@ -141,7 +141,7 @@ The above output indicates that the installation completed successfully and poin
7. Now let's search for our module:

> [!NOTE]
> To make your custom modules appear in `module avail`, `module spider` etc., you can set the `$CCR_CUSTOM_BUILD_PATHS` environment variable following the instructions outlined [here](https://docs.ccr.buffalo.edu/en/latest/software/building/#building-modules-for-your-group).
> To make your custom modules appear in `module avail`, `module spider` etc., you can set the `$CCR_CUSTOM_BUILD_PATHS` environment variable following the instructions outlined in [CCR's module building documentation](https://docs.ccr.buffalo.edu/en/latest/software/building/#building-modules-for-your-group).
> To ensure the path gets picked up on login, point it to `~/.ccr/modulepaths`.

```
Expand Down
2 changes: 1 addition & 1 deletion easybuild/2_ApplicationSpecific/gaussian/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ eb Gaussian-16.C.02-AVX2.eb --umask=007
## Using your module

> [!NOTE]
> To make your custom modules appear in `module avail`, `module spider` etc., you can set the `$CCR_CUSTOM_BUILD_PATHS` environment variable following the instructions outlined [here](https://docs.ccr.buffalo.edu/en/latest/software/building/#building-modules-for-your-group).
> To make your custom modules appear in `module avail`, `module spider` etc., you can set the `$CCR_CUSTOM_BUILD_PATHS` environment variable following the instructions outlined in [CCR's module building documentation](https://docs.ccr.buffalo.edu/en/latest/software/building/#building-modules-for-your-group).
> To ensure the path gets picked up on login, point it to `~/.ccr/modulepaths`.

Once the installation completes successfully, when you search for the module you should see it listed in a section at the top with your group's installation path listed. For example:
Expand Down
2 changes: 1 addition & 1 deletion easybuild/2_ApplicationSpecific/orca/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ module load easybuild
2. Upload your Orca software installation file to CCR and place in your home or project directory. We recommend downloading the x86_64, `.tar.xv` archive version.

3. Create Orca Easybuild recipe
In the folder that your Orca software was uploaded to, create a file called `ORCA-6.0.1-gompi-2021b.eb` Copy the contents from [this example](ORCA-6.0.1-gompi-2021b.eb) Easybuild recipe and place them in your new recipe file. Edit the file as described, save and exit the editor. There are several things to be aware of with this recipe file:
In the folder that your Orca software was uploaded to, create a file called `ORCA-6.0.1-gompi-2021b.eb` Copy the contents from [CCR's ORCA Easybuild recipe example](ORCA-6.0.1-gompi-2021b.eb) and place them in your new recipe file. Edit the file as described, save and exit the editor. There are several things to be aware of with this recipe file:
- You MUST name the file in the same way that CCR provides in this example
- If you're installing a different version of Orca, you must update the Easybuild recipe name with the updated version number AND change the version number within your Easybuild recipe on line 7.
- You will need to update the checksum value on line 20 to match the checksum of your software installation media. To get the checksum of a file, run: `sha256sum filename`
Expand Down
Loading