Update Derecho site config

[climbfuji]

Description

1. Update Derecho site configuration to newer compiler versions, use native crayenv instead of ncarenv.
2. Submodule pointer update for spack to fix an issue on Derecho - never attempt to recursively scan the root filesystem (/). See spack PR filesystem.py: prevent recursively traversing the root filesystem spack#576 for more information and links to upstream spack discussion.

Dependencies

• Waiting on filesystem.py: prevent recursively traversing the root filesystem spack#576

Issues addressed

Working toward #1883

Applications affected

None

Systems affected

Derecho

Testing

• CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
  • GitHub actions CI tests pass
  • GitHub actions CI tests do not pass (provide explanation)
  • GitHub actions CI tests skipped (provide explanation if necessary)
• New tests added: List and describe any new tests added to GitHub actions
  • ...
• Additional testing: Add information on any additional tests conducted
  • Built oneAPI and GCC stacks on Derecho (unified environment). Tried jedi-bundle: Compiled and ran tests (skipped MPI tests that required qsub) with GCC. Compiled and ran into the well-known error in MPAS with oneAPI (but cmake step completed and most of the compilation succeeded as well). This is as far as I can go.

Checklist

• This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
• These changes have been tested on the affected systems and applications.
• All dependency PRs/issues have been resolved and this PR can be merged.
• All necessary updates to the documentation (spack-stack wiki) will be made when this PR is merged

Wiki update for Derecho - will be made when PR is merged and spack-stack-2.1 user documentation is prepared

1. How to build spack-stack environments. Nothing special to do, happens automatically when running source setup.sh
2. How to use spack-stack environments:

# GENERAL

module --force purge

module use /opt/cray/pe/modulefiles
module use /glade/work/epicufsrt/contrib/spack-stack/derecho/installs/oneapi-2025.3.1/modulefiles


# ONEAPI

module use /lustre/desc1/scratch/heinzell/spst-20260209/envs/ue-oneapi-2025.3.1/modules/Core

module load crayenv/25.03
module load PrgEnv-intel/8.6.0

module load stack-intel-oneapi-compilers/2025.3.1
module load stack-cray-mpich/8.1.32

module unload cray-libsci

module available


# GNU

module use /lustre/desc1/scratch/heinzell/spst-20260209/envs/ue-gcc-13.3.1/modules/Core

module load crayenv/25.03
module load PrgEnv-gnu/8.6.0

module load stack-gcc/13.3.1
module load stack-cray-mpich/8.1.32

module unload cray-libsci

module available

[rickgrubin-noaa]

Pleased to see that crayenv is available so as to be consistent with (from EPIC's perspective) gaea-c6.

A couple of comments for posterity:

• oneapi@2025.3.1 is nice to have; its installation does place an onus on EPIC to maintain it, which is not necessarily desireable -- preference is for system tools to be installed by parties responsible for the system
• a source cache is also nice to have, and has been discussed for other EPIC hosts. There are disk space constraints to keep in mind, as it's not always clear whether or not EPIC's allocations can be increased

[climbfuji]

Pleased to see that crayenv is available so as to be consistent with (from EPIC's perspective) gaea-c6.

A couple of comments for posterity:

• oneapi@2025.3.1 is nice to have; its installation does place an onus on EPIC to maintain it, which is not necessarily desireable -- preference is for system tools to be installed by parties responsible for the system

I would argue that maintaining this is so little work, maybe even less than dealing with what CISL throws at you.

• a source cache is also nice to have, and has been discussed for other EPIC hosts. There are disk space constraints to keep in mind, as it's not always clear whether or not EPIC's allocations can be increased

I didn't add it, it was already there. I didn't populate it yet either. But happy to remove it if you think it's better.

[rickgrubin-noaa]

• oneapi@2025.3.1 is nice to have; its installation does place an onus on EPIC to maintain it, which is not necessarily desireable -- preference is for system tools to be installed by parties responsible for the system

I would argue that maintaining this is so little work, maybe even less than dealing with what CISL throws at you.

I would argue that as well; I am obliged to represent EPIC mgmt's decisions and point of view.

• a source cache is also nice to have, and has been discussed for other EPIC hosts. There are disk space constraints to keep in mind, as it's not always clear whether or not EPIC's allocations can be increased

I didn't add it, it was already there. I didn't populate it yet either. But happy to remove it if you think it's better.

I was not aware that the source cache was already there; given that, let's update it per the latest envs.