Cray MPICH

Installation
Compiling Applications
Running Containerized MPI Applications
Differences from CPE

For detailed documentation, refer to the Cray MPICH 9 documentation.

Cray MPICH is installed inside the container only. There is nothing to install on the host system.

Installation

Cray MPICH is installed by default in the SUPS reference container images. The following compiler-specific Cray MPICH variants are available in the SUPS reference container images:

Value	Compiler suite	Available in stack
`amd`	ROCm LLVM compilers (`amdclang`, `amdclang++`, `amdflang`)	AMD
`aocc`	AMD Optimizing C/C++ and Fortran Compilers (AOCC)	AMD
`cray`	Cray Compiler Environment (CCE)	AMD, NVIDIA
`gnu`	GNU compilers	AMD, NVIDIA
`nvidia`	NVIDIA HPC compilers (`nvc`, `nvc++`, `nvfortran`)	NVIDIA

To install Cray MPICH manually in a development container, follow the run-time container template to set up all HPE RPM repositories (cpe, shs, shmem, uss, and sups-oss) and then install the Cray MPICH variant that matches your compiler of choice with the corresponding package manager.

For example, to install the Cray MPICH 9.2.0 development package for the ROCm LLVM compilers in the AMD Rocky Linux 9.6 development container:

dnf install cray-mpich-9.2.0-amd713-devel

The Cray MPICH development package name follows the format cray-mpich-${VERSION}-${COMPILER}${COMPAT}-devel where VERSION is the Cray MPICH version (e.g. 9.2.0), COMPILER is the compiler suite suffix (e.g. amd, aocc, cray, gnu, nvidia), and COMPAT is the compiler compatibility version string. To find the exact name of the Cray MPICH development package for your compiler, you can search the package manager with

# For Rocky Linux 9.6 development container
dnf search cray-mpich

# For SLES 15 SP7 development container
zypper search cray-mpich

The Cray MPICH installation path is /opt/cray/pe/mpich/${VERSION}/ofi/${COMPILER}/${COMPAT}. For example, here are the installation paths for the Cray MPICH 9.2.0 packages in a NVIDIA SUSE Linux BCI 15.7 container:

# Cray MPICH 9.2.0 for CCE
/opt/cray/pe/mpich/9.2.0/ofi/cray/20.0

# Cray MPICH 9.2.0 for GCC
/opt/cray/pe/mpich/9.2.0/ofi/gnu/12.3

# Cray MPICH 9.2.0 for NVIDIA HPC Compilers
/opt/cray/pe/mpich/9.2.0/ofi/nvidia/26.3

[!important] Cray MPICH must be used with the compiler it was built for. Please refer to each compiler’s own SUPS documentation to install and configure the compilers in the container before using Cray MPICH. For example, if you are using the Cray MPICH 9.2.0 for GNU, you must first follow the GCC documentation to install and configure the GCC 14 in the container before using Cray MPICH.

Compiling Applications

It is recommended to compile and link MPI applications with Cray MPICH using the appropriate MPI compiler wrappers (mpicc, mpicxx, mpifort) that are provided by Cray MPICH.

[!important] In the SUPS Rocky containers, the default GCC version is the version provided by the base operating system. This version differs from the GCC 14 toolchain used to build and validate the HPE Cray MPICH GNU libraries.

To use GCC Toolset 14, first enable its environment:
source /opt/rh/gcc-toolset-14/enable
After enabling GCC Toolset 14, the default gcc, g++, and gfortran commands will refer to the GCC 14 compiler suite.

The HPE Cray MPICH compiler wrappers (mpicc, mpicxx, and mpifort) invoke the GNU compilers using their standard command names. Users are encouraged to enable GCC Toolset 14 before compiling applications intended to use the GNU-based HPE Cray MPICH libraries when using the SUPS Rocky containers.

For example, if you are using the Cray MPICH 9.2.0 for CCE 22.0.0 in the NVIDIA SUSE Linux BCI 15.7 container, you can compile your Fortran MPI application with the following command.

/opt/cray/pe/mpich/9.2.0/ofi/cray/20.0/bin/mpifort -o my_mpi_app my_mpi_app.f90

Please refer to Cray MPICH’s documentation about the usage of the MPI compiler wrappers.

Running Containerized MPI Applications

SUPS provides the hpc-container-wrapper utility to run containerized MPI applications with Cray MPICH on HPE Slingshot systems. Please refer to the SUPS Utility Tools documentation for more details.

Differences from CPE

If you are transitioning from a Cray Programming Environment (CPE) system, here are some of the differences when using Cray MPICH in the SUPS container environment.

With the CPE module environment, you can load the compiler module PrgEnv-* to choose a compiler suite to use, load the CrayPE module craype to enable the CrayPE compiler wrappers, and then load the Cray MPICH module cray-mpich to use the Cray MPICH built for the selected compiler suite. Optional GPU support can be enabled by loading the craype-accel-* module for the corresponding GPU architecture. Then the CrayPE compiler wrappers (e.g. cc, CC, ftn) will automatically use the correct Cray MPICH variant and include the correct header and library search paths and libraries based on the loaded modules.

In the SUPS container environment, there is no module environment. The CrayPE compiler wrappers are no longer available. You need to find the Cray MPICH installation path for your compiler of choice and then use the Cray MPICH wrapper compilers to compile MPI applications.