RELEASED: HPE Cray Programming Environment 24.07 General Availability Release for HPE Cray Supercomputing EX and HPE Cray Supercomputing Systems with CSM 1.5 COS 24.7 SLES15 SP5 - Rev.A – August 2024

PRODUCT DESCRIPTION

The HPE Cray Programming Environment 24.07 release for HPE Cray Supercomputing EX and HPE Cray Supercomputing systems is now available for CSM 1.5 COS 24.7 SLES15 SP5 systems.

COS 24.7 consists of COS Base 3.1.0/USS 1.1.0.
NOTE: COS 2.4 (based on SLES 15 SP4) is not supported in this release.
COS 2.4 users must use the CPE 23.09 (or earlier) release.

SOFTWARE OVERVIEW

This software announcement applies to HPE Cray Supercomputing EX and/or HPE Cray Supercomputing CSM 1.5 COS 24.7 SLES15 SP5 systems.

HPE Cray Programming Environment licensed customers may download the release materials by submitting a request to HPE PointNext for the package below (depending on entitlement):

HPE Cray Programming Environment 24.07 for CSM

CPE support for specific HPE Cray Supercomputing EX recipes may be found in the following documents which are maintained for each recipe on the HPE Support Center:

HPE Cray EX System Software Stack Installation and Upgrade Guide for CSM (S-8052)
HPE Cray Software Compatibility Matrixes Reference Guide (S-8042)

See “module help <product>” for the corresponding PE product release notes.

Content for CSM 1.5 COS 24.7 SLES15 SP5 Systems:

cpe-24.07-sles15-sp5-csm-24.7.2.tar.gz

NOTE: The single tarfile(s) above contain base PE as well as optional installation of other third-party compiler support.

REQUIRED: Base PE
OPTIONAL:

AMD AOCC 4.2.0

AMD ROCm 6.1

Intel OneAPI 2024.0.2

NVIDIA HPC SDK 24.03

CUDA 12.3

GCC 13

Documentation

(NOTE: Documentation**, including revision documentation,** may be found on support.hpe.com; plus many CPE docs are also released in a tar file as part of the official release package as well. To find a specific document on support.hpe.com, simply do a search on “pubno 24.07” without the quotation marks. For example, search on S-8003 24.07 to find the CPE 24.07 version of the S-8003 document.)

Installation Guide

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (24.07) S-8003

HPE_CPE_Installation_Guide_CSM_on_HPE_Cray_Supercomputing_EX_Systems_24.07_S-8003.pdf

User Guide

HPE Cray Programming Environment User Guide: CSM on HPE Cray Supercomputing EX Systems (24.07) S-8005

HPE_CPE_User_Guide_CSM_on_HPE_Cray_Supercomputing_EX_Systems_24.07_S-8005.pdf

Software Supported

The following software is supported with this release:

Cluster System Manager (CSM) 1.5 - managed systems running SLES15 SP5 on the compute/login nodes.
COS 24.7 components comprise:
- COS Base
- HPE Cray Supercomputing User Services Software (USS)
- HPE SUSE Linux Enterprise Server

IMPORTANT NOTES

NOTE: Hidden symbol errors when linking Fortran with CCE 18.0.0 Linking Fortran applications may fail with an error message of the form “hidden symbol `<SYMBOL>’ in <LIB> is referenced by DSO”. If this error message is seen, it can usually be worked around by adding ‘-lgcc_s’ to your link line. This is known to affect use of craypat and cray-parallel-netcdf but may be seen without them as well.

NOTE: NVIDIA HPC SDK releases include three versions of CUDA. For each new SDK version supported by HPE, Cray PE supports only the latest CUDA version included in the respective NVIDIA HPC SDK release (i.e., the older two CUDA versions are not supported by CPE when a new SDK is supported).

NOTE: CUDA 12.0 is compatible with GCC 11 (and below).

NOTE: Starting with CPE 23.12, cray-gcc packages will no longer be provided for SLES and COS based systems. Instead, SLES and COS based systems must use SLES-provided gcc*, gcc*-c++, and gcc*-fortran packages (available from the SLES Development Tools Module). Customers who wish to continue to use cray-gcc should continue to use CPE 23.09 or earlier CPE releases.

NOTE: Starting with CPE 23.12, the ROCm modulefile can now be created with craypkg-gen. The generated implementation of the Lmod ROCm modulefile works with the CPE amd modulefile.

NOTE: Perftools 24.07.0 and PAPI 7.1.0.2 included with CPE 24.07 are not compatible with ROCm earlier than 6.0.0, they require the use of ROCm 6.0.0 or later.

NOTE: Starting with CPE 24.03, on systems with aarch64 CPUs and NVIDIA GPUs, if
MPICH_GPU_SUPPORT_ENABLED=1 is set, HPE Cray MPI automatically chooses to disable the use of XPMEM for intra-node, inter-process MPI data movement operations that involve memory regions that are managed by system allocators (e.g., mmap, malloc, and new). This is done to guard against potential interactions between XPMEM and the GPU runtime layer that may cause node failures to occur. Instead of using XPMEM, HPE Cray MPI uses Linux Cross Memory Attach (CMA) to optimize these MPI operations. It is important to note that GPU Peer2Peer IPC will continue to be used for intra-node, inter-process data movement operations involving memory regions managed via GPU memory allocators (e.g., cudaMalloc, cudaFree).

NOTE: CPE 24.03 is not supported on XD670.

NOTE: Beginning in CPE 24.07, the NVHPC modules (PrgEnv-nvhpc, nvhpc, nvhpc-mixed) contain deprecation messages. The NVHPC modules will be removed at a future release in favor of the NVIDIA modules (PrgEnv-nvidia, nvidia, nvidia-mixed). The move towards the NVIDIA modules is to complete the alignment of CPE module flows. The module flow for all environments is as follows:

Load an environment meta module (ie. PrgEnv-nvidia)

Environment meta module loads a compiler (ie. nvidia)

User can choose to load a toolkit (cuda, cudatoolkit)

 Please note that the new “cuda” module is being released as beta. This new module 
 is generated along with CPE’s rocm toolkit and third-party compiler modules via 
 craypkg-gen.

NOTE: In CPE 24.07 a late issue was located on EX254 and EX235n for NVIDIA TCL modules. This issue impacts NVIDIA machines using the TCL Modules: NVHPC and CUDA. To resolve this, a site admin should insert the following code snippet at the end of the gen_modulefiles.sh script:

     for cuda_module in ls /opt/modulefiles/cuda/; do
       sed -i "s/$SDK_LEVEL_$MOD_LEVEL/$SDK_LEVEL\_$MOD_LEVEL/g" /opt/modulefiles/cuda/${cuda_module}
       done
    
     for nvhpc_module in ls /opt/modulefiles/nvhpc/; do
       sed -i -E "s/Linux_(.*64)$/Linux_\1/$NVHPC_LEVEL/" /opt/modulefiles/nvhpc/${nvhpc_module}
     done

 On CSM systems, the script is located in the cpe-config-management repository under 
 roles/cray.pe-deploy/files/gen_modulefiles.sh.

CPE 24.07 Product Versions:

NOTE: Use of **** indicates a new or updated component version compared to CPE 24.03.

HPE Cray Programming Environment for HPE Cray Supercomputing EX and HPE Cray Supercomputing Systems with CSM


Cray Compiling Environment – CCE
cce 18.0.0 ****

Cray Message Passing Toolkit - CMPT
cray-mpich 8.1.30 ****
cray-mpixlate 1.0.5 ****
cray-dsmml 0.3.0
cray-pmi 6.1.15 ****
cray-openshmemx 11.7.2 ****

Application Launch Tools - ALT
cray-pals 1.3.2

Cray Debugging Support Tools – CDST
cray-cti 2.18.4 ****
gdb4hpc 4.16.2 ****
cray-ccdb 5.0.4 ****
cray-stat 4.12.3 ****
atp 3.15.4 ****
valgrind4hpc 2.13.3 ****
sanitizers4hpc 1.1.3 ****
cray-dyninst 12.3.2 ****
cray-mrnet 5.1.3 ****

Cray Performance Measurement & Analysis Tools – CPMAT
perftools 24.07.0 ****
cray-papi 7.1.0.2 ****

Cray Scientific and Math Libraries - CSML
cray-libsci 24.07.0 ****
cray-libsci-acc 24.07.0 ****
cray-fftw 3.3.10.8 ****

Cray Deep Learning Tools
craype-dl-plugin-ftr 22.06.1.2
craype-dl-plugin-py3 24.03.1

Cray Environment Setup and Compiling support – CENV
craypkg-gen 1.3.33 ****
craype 2.7.32 ****
cpe-prgenv 8.5.0
cray-lmod 8.7.37 ****
cray-modules 3.2.11.7

Third party products
cray-hdf5 1.14.3.1 ****
cray-netcdf 4.9.0.13 ****
cray-parallel-netcdf 1.12.3.13 ****
cray-python 3.11.7
cray-R 4.4.0 ****

Third-party products supported
Totalview 2024.1.21
Forge 23.1.2

*******************************************************************************************

Certain components, files or programs contained within this package or product are Copyright - 2024 Hewlett Packard Enterprise Development LP. All trademarks used in this document are the property of their respective owners.