RELEASED: HPE Cray Programming Environment 24.11 Release for HPE Cray XD2000 Gen 10 Plus Systems (x86) - Rev.1 – January 2025

PRODUCT DESCRIPTION

The HPE Cray Programming Environment 24.11 release is now available for HPE Cray XD2000 Gen 10 Plus systems (x86) running RHEL 8.10 with HPCM 1.12.

NOTE: To better align with HPE offerings, software and documentation files previously known as “Apollo 2000” will now be identified as “HPE Cray XD2000”. Previous CPE release versions will remain on the HPE Software support Center unchanged. The change will only affect 24.11 and forward releases.

SOFTWARE OVERVIEW

The HPE Cray Programming Environment 24.11 release for HPE Cray XD2000 Gen10 Plus (x86) consists of the products/components listed further below.

See “module help <product>” for the corresponding PE product release notes.

Content for HPE Cray XD2000 Systems:

CPE-HPE-CRAY-XD2000-24.11-05.el8.x86_64.iso

Software Supported

The following software is supported with this release:

HPE Performance Cluster Manager (HPCM) 1.12 - managed systems running RHEL 8.10 on the compute/login nodes.

Workload management system options:

Slurm 22.05.8 (or later)

PBS Professional 2021.1.1 (or later)

AMD AOCC 4.2.0

AMD ROCm 6.2.1

Intel OneAPI 2024.2.0

NVIDIA HPC SDK Unsupported

GCC 13

MOFED 5.8

Documentation

NOTE: As of CPE 24.11 and later, HPE discontinued providing CPE release announcement and user guide documentation on the HPE Support Center web page. CPE 24.11 (and later) release announcement and user guides, along with installation guides, are now available on the CPE Documentation site. CPE installation guides for all CPE versions can still be accessed from the HPE Support Center web page. CPE 24.07 and earlier documentation, including user guides, installation guides, release announcements, and related revision documentation, can also be found on the HPE Support Center web page.

Installation Guide

HPE Cray Programming Environment Installation Guide: HPE Cray XD2000 Gen10 Plus (x86) Systems (24.11)

HPE_CPE_Installation_Guide_HPE_Cray_XD2000_Gen10_Plus_x86_Systems_24.11.pdf

User Guide

HPE Cray Programming Environment User Guide: HPE Cray XD2000 Gen10 Plus (x86) Systems (24.11)

HPE_CPE_User_Guide_HPE_Cray_XD2000_Gen10_Plus_x86_Systems_24.11.pdf

IMPORTANT NOTES

NOTE: Update to HPE Cray XD2000 Gen 10 Plus Setup Scripts. The setup scripts for HPE Cray XD2000 Gen 10 Plus systems, located in the cray-pe-profile package, have been updated to look for CPE’s provided Lmod package inside the CPE namespace (/opt/cray/pe). The setup scripts are responsible for how CPE behaves on a system. Please use caution when installing, so that the systems current settings in /etc/cray-pe.d/cray-pe-configuration.sh and /etc/cray-pe.d/cray-pe-configuration.csh are not lost.

NOTE: MPIs other than cray-mpich may not work on systems with Cray Programming Environment installed.

NOTE: Potential issue with libfabric and MOFED 5.8. When using the OFI netmod of Cray-MPICH there is a potential issue when calling MPI_Finalize which causes the application to seg fault. This is seen at 64 ranks and larger. If encountered, setting the environment variable FI_VERBS_PREFER_XRC=0 will allow for applications to finish. Note that there is a significant performance penalty at scale when setting this environment variable.

NOTE: The Red Hat GCC Toolset provides the cc compiler driver that will conflict with the craype driver. To ensure the environment’s PATH is correct, the craype module should always be reloaded after swapping or reloading the gcc-native module. You may run cc –craype-help to verify the correct executable is resolved.

NOTE: Perftools 24.11.0 and PAPI 7.1.0.4 included with CPE 24.11 are not compatible with ROCm earlier than 6.0.0, they require the use of ROCm 6.0.0 or later.

NOTE: Beginning in CPE 24.11, the NVHPC modules (PrgEnv-nvhpc, nvhpc, nvhpc-mixed) are no longer offered. The NVIDIA modules (PrgEnv-nvidia, nvidia, nvidia-mixed) are now the sole option for modules regarding the NVIDIA programing environment. The move to the NVIDIA modules is to complete the alignment of CPE module flows. The module flow for all environments is as follows:

Load an environment meta module (ie. PrgEnv-nvidia)

Environment meta module loads a compiler (ie. nvidia)

User can choose to load a toolkit (cuda, cudatoolkit)

NOTE: 24.11/ 25.03 HPE HPC Software 2025 roadmap update for HPE Cray systems with NVIDIA CPUs and GPUs. The NVIDIA driver support for SLES15 SP6 was not available before code completion dates, as a result, the January 2025 HPE HPC continuous software stack releases are for HPE Cray EX systems without NVIDIA CPUs and GPUs. For HPE Cray EX Systems with NVIDIA CPUs and GPUs, please use the August 2024 (CPE 24.07) HPE HPC continuous software stack. These software stacks were validated with NVIDIA HPC SDK 24.3. The March 2025 HPE HPC continuous and extended software stack releases will be validated with NVIDIA HPC SDK 24.11 and will support all HPE CRAY EX systems.

CPE 24.11 Product Versions:

NOTE: The use of *** indicates a new or updated component version compared to CPE 24.07.

HPE Cray Programming Environment for HPE Cray XD2000 Gen 10 Plus Systems


Cray Compiling Environment – CCE
cce 18.0.1 ****

Cray Message Passing Toolkit - CMPT
cray-mpich 8.1.31 ****
cray-dsmml 0.2.2
cray-pmi 6.1.15

Application Launch Tools - ALT
cray-pals 1.3.2

Cray Debugging Support Tools – CDST
cray-cti 2.19.0 ****
gdb4hpc 4.16.3 ****
cray-ccdb 5.0.5 ****
cray-stat 4.12.4 ****
atp 3.15.5 ****
valgrind4hpc 2.13.4 ****
sanitizers4hpc 1.1.4 ****
cray-dyninst 12.3.4 ****
cray-mrnet 5.1.4 ****
cray-zmqnet 1.0.0 ****

Cray Performance Measurement & Analysis Tools – CPMAT
perftools 24.11.0 ****
cray-papi 7.1.0.4 ****

Cray Scientific and Math Libraries - CSML
cray-libsci 24.11.0 ****
cray-fftw 3.3.10.9 ****

Cray Deep Learning Tools
craype-dl-plugin-py3 21.02.1.3

Cray Environment Setup and Compiling support – CENV
craypkg-gen 1.3.34 ****
craype 2.7.33 ****
cpe-prgenv 8.4.0
cray-lmod 8.7.31
cray-modules 3.2.11.5

Third party products
cray-hdf5 1.14.3.3 ****
cray-netcdf 4.9.0.15 ****
cray-parallel-netcdf 1.12.3.15 ****
cray-python 3.11.7
cray-R 4.4.0

Third-party products supported
Totalview 2024.1.21

*******************************************************************************************