HPE Cray Supercomputing Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (26.03-Rev. A) S-8003

About the HPE CPE Installation Guide: CSM on HPE Cray Supercomputing EX Systems

The HPE Cray Supercomputing Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (S-8003) contains procedures for installing the HPE Cray Supercomputing Programming Environment (CPE) and third-party programming environment components, including Perforce TotalView, Linaro Forge, and AMD and Intel compilers.

This publication is intended for system administrators who want to:

  • Install or reinstall all CPE components,

  • Install additional licensed components, or

  • Customize the programming environment files before use.

This guide assumes that users have:

  • Familiarity with standard Linux and open source tools, including Ansible, YAML, and (optionally) Kubernetes, and

  • Access to a method of obtaining the most current CPE tar files.

IMPORTANT: Be sure to always employ the latest installation instructions. To check for the latest version and revisions of this CPE guide, go to the HPE Support Center website, and perform a search on the part number of this document (S-8003).

For information on how to use CPE or details regarding CPE components and modules, see the CPE Online Documentation website web page. See also the Documentation and support chapter for additional CPE resources and information.

Release information

This publication supports the installation of CPE 26.03 on HPE Cray Supercomputing EX systems with specific configurations:

Management Software & Version

COS Version

Operating System

Architecture

GCC Version

CSM 1.7.1

USS 1.5.X

SLES 15 SP7

X86

14.0

CSM 1.7.1

USS 1.5.X

SLES 15 SP7

AArch64

14.0

CSM 1.7.1

USS 1.5.X

SLES 15 SP6

X86

14.0

CSM 1.7.1

USS 1.5.X

SLES 15 SP6

AArch64

14.0

This release also supports v21.0.0 of the HPE Cray Compiler Environment (CCE).

IMPORTANT: In releases before the HPE Cray Supercomputing Operating System (COS) 25.9 release, COS comprised three components: COS Base, HPE USS, and HPE SLES. Starting with this CPE 25.09 release, COS 25.9 (and later) comprises:

  • HPE Cray Supercomputing User Services Software (USS)

  • HPE SUSE Linux Enterprise Server (SLES)

See the CPE 26.03 Release Announcements on the CPE Online Documentation website for other supported dependencies.

Variable substitutions

Use the following variable substitutions throughout the included procedures.

  • <CPE_RELEASE> = 26.03

  • <CPE_VERSION> = 26.03.X (where X is the third digit of the actual three-digit version number of an official and HPE-supported version number)

  • <spX> or <SPX> = SP6 (as appropriate)

  • <spX> or <SPX> = SP7 (as appropriate)

Supporting documentation

References to specific and related HPE Cray Supercomputing EX documentation are found throughout this guide. Direct links to these references (for supported COS/CSM releases defined in Installation Prerequisites) are provided in HPE Cray Supercomputing EX documentation links.

Record of revision

New in the CPE 26.03 (Rev. A) publication

New in the CPE 26.03 publication

New in the CPE 25.09 (Rev. A) publication

New in the CPE 25.09 publication

New in the CPE 25.03 publication

New in the CPE 24.11 (Rev. 2) publication

New in the CPE 24.11 (Rev. 1) publication

New in the CPE 24.11 publication

New in the CPE 24.07 publication

New in the CPE 24.03 publication

Publication Title

Date

HPE Cray Supercomputing Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (26.03-Rev. A) S-8003

April 2026

HPE Cray Supercomputing Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (26.03) S-8003

March 2026

HPE Cray Supercomputing Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (25.09-Rev. A) S-8003

December 2025

HPE Cray Supercomputing Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (25.09) S-8003

September 2025

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (25.03) S-8003

March 2025

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (24.11-Rev. 2) S-8003

January 2025

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (24.11-Rev. 1) S-8003

January 2025

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (24.11) S-8003

January 2025

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems (24.07) S-8003

August 2024

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (24.03) S-8003

May 2024

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (23.12) S-8003

December 2023

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (23.09) S-8003

September 2023

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (23.05) S-8003

June 2023

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (23.02 Rev A) S-8003

March 2023

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (23.02) S-8003

February 2023

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.12) S-8003

December 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.11) S-8003

November 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.10) S-8003

October 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.09) S-8003

September 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.08 Rev A) S-8003

August 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.08) S-8003

August 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.06 Rev A) S-8003

July 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.06) S-8003

June 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.05) S-8003

May 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.04) S-8003

April 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.03) S-8003

March 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (22.02) S-8003

February 2022

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (21.12) S-8003

December 2021

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (21.11) S-8003 Rev A

November 2021

HPE Cray Programming Environment Installation Guide: CSM on HPE Cray EX (21.11) S-8003

November 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.10) S-8003

October 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.09) S-8003

September 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.08) S-8003

August 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.07) S-8003 Rev A

July 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.07) S-8003

July 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.06) S-8003

June 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.05) S-8003

May 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.04) S-8003

April 2021

HPE Cray Programming Environment Installation Guide: CSM 1.4 on HPE Cray EX (21.03) S-8003

March 2021

HPE Cray Asynchronous Installer Guide (21.03) S-8003

March 2021

Document conventions

This section defines the documentation conventions used throughout the guide, including typographic styles for code, commands, paths, and the backslash as the shell line-continuation character. It explains command-prompt notation, showing how the host and account are indicated (root prompts end with #, non-root prompts use account@hostname>) and lists node abbreviations (CN, NCN, AN, UAN) with example prompts for specific node types and Kubernetes contexts. This section also provides a simple three-step workflow and a reminder to verify pasted commands.

Typographical and command prompt conventions

This section provides background information about typographical and command prompts used in this guide and describes how they are delineated throughout this guide.

Typographical conventions

Type

Convention Description

This style

Indicates program code, reserved words, library functions, command-line prompts, screen output, file/path names, variables, and other software constructs.

\ (backslash)

When inserted at the end of a command line, indicates the Linux shell line continuation character (lines joined by a backslash are parsed as a single line).

Command prompt conventions

Host name and account in command prompts: The host name in a command prompt indicates where the command must be run. The account that must run the command is also indicated in the prompt.

  • The root or super-user account always has the # character at the end of the prompt.

  • Any non-root account is indicated with account@hostname>. A user account that is not root or crayadm is seen as user.

Command Prompt

Definition

user@login>

Run the command on any login node as any non-root user.

hostname#

Run the command on the specified system as root.

user@hostname>

Run the command on the specified system as any non-root user.

Copying and pasting text from this document

Using the Copy and Paste functions from a PDF is unreliable. Although copying and pasting a command line typically works, copying and pasting formatted file content (for example, JSON, YAML) typically fails. To ensure that file content is copied and pasted correctly while performing the procedures in this guide:

  1. Copy the content from the PDF.

  2. Paste it to a neutral editing form and add the necessary formatting.

  3. Copy the content from the neutral form and paste it into the console.

Tip: As a best practice, double-check copied/pasted commands for correctness, as some commands may not render correctly in the PDF.

Downloading HPE Cray Supercomputing EX software

To download HPE Cray Supercomputing EX software, refer to the HPE Support Center or download it directly from My HPE Software Center. The HPE Support Center contains a wealth of documentation, training videos, knowledge articles, and alerts for HPE Cray Supercomputing EX systems. It provides the most detailed information about a release as well as direct links to product firmware, software, and patches available through My HPE Software Center.

Downloading the software through the HPE Support Center

HPE recommends downloading software through the HPE Support Center because of the many other resources available on the website.

  1. Visit the HPE Cray Supercomputing EX product page on the HPE Support Center.

  2. Search for specific product info, such as the full software name or recipe name and version.

    For example, search for “Slingshot 2.1” or “Cray System Software with CSM 24.3.0.”

  3. Find the desired software in the search results and select it to review details.

  4. Select Obtain Software and select Sign in Now when prompted.

    If a customer’s Entitlement Order Number (EON) is tied to specific hardware rather than software, the software is available without providing account credentials. Access the software instead by selecting Download Software and skip the next step in this procedure.

  5. Enter account credentials when prompted and accept the HPE License Terms.

    To download software, customers must ensure their Entitlement Order Number (EON) is active under My Contracts & Warranties on My HPE Software Center. If customers have trouble with the EON or are not entitled to a product, they must contact their HPE contract administrator or sales representative for assistance.

  6. Choose the needed software and documentation files to download and select curl Copy to access the files.

    Just like the software files, the documentation files change with each release. In addition to the official documentation, valuable information for a release is often available in files that include the phrase README in their name. Be sure to select and review these files in detail.

    HPE recommends the curl Copy option, which downloads a single text file with curl commands to use on the desired system. You must run the curl commands within 24 hours of downloading them or download new commands if more than 24 hours have passed.

    To validate the security of the downloads, you can later compare the files on the desired system against the checksums provided by HPE underneath each selected download.

  7. Save the text file to a central location.

  8. On the system where the software will be downloaded, run a shell script to execute the text file that includes the curl commands.

    For example:

    ncn-m001# bash -x <TEXT_FILE_PATH>

    The -x option in this example tracks the download progress of each curl command in the text file.

Downloading the software directly from the My HPE Software Center

Users already familiar with a release can save time by downloading software directly from My HPE Software Center.

  1. Visit My HPE Software Center and select Sign in.

  2. Enter account credentials when prompted and select Software in the left navigation bar.

  3. Search for specific product info, such as the full software name or recipe name and version.

    For example, search for “Slingshot 2.1” or “Cray System Software with CSM 24.3.0.”

  4. Find the desired software in the search results and review details by selecting Product Details under the Action column.

    Image of Product Detailsoption

  5. Select Go To Downloads Page and accept the HPE License Terms.

    To download software, customers must ensure their Entitlement Order Number (EON) is active under My Contracts & Warranties. If customers have trouble with the EON or are not entitled to a product, they must contact their HPE contract administrator or sales representative for assistance.

  6. Choose the needed software and documentation files to download and select curl Copy to access the files.

    Just like the software files, the documentation files change with each release. In addition to the official documentation, valuable information for a release is often available in files that include the phrase README in their name. Be sure to select and review these files in detail.

    HPE recommends the curl Copy option, which downloads a single text file with curl commands to use on the desired system. You must run the curl commands within 24 hours of downloading them or download new commands if more than 24 hours have passed.

    To validate the security of the downloads, you can later compare the files on the desired system against the checksums provided by HPE underneath each selected download

  7. Save the text file to a central location.

  8. On the system where the software will be downloaded, run a shell script to execute the text file that includes the curl commands.

    For example:

    ncn-m001# bash -x <TEXT_FILE_PATH>

    The -x option in this example tracks the download progress of each curl command in the text file.

About Ansible

Ansible is an open-source software provisioning and configuration management tool. The CPE Installer leverages Ansible playbooks and roles to install CPE components. Below is an example of the pe_deploy.yml playbook:

---
- hosts: uai:Application_UAN:Application:Compute
  any_errors_fatal: true
  gather_facts: no
  remote_user: root
 
  pre_tasks:
    - name: Unmount any overlays first
      command: bash /etc/cray-pe.d/pe_overlay.sh cleanup
      when:
        - not cray_cfs_image
        - forcecleanup | default(false)
      ignore_errors: yes
 
  roles:
    - { role: cray.pe_deploy, cray_pe_pkg: aocc, when: not cray_cfs_image }
    - { role: cray.pe_deploy, cray_pe_pkg: intel, when: not cray_cfs_image }
    - { role: cray.pe_deploy, when: not cray_cfs_image }
 
  post_tasks:
    - name: Run mount overlay setup script
      command: bash /etc/cray-pe.d/pe_overlay.sh
      when:
        - not cray_cfs_image
        - not forcecleanup | default(false)

IMPORTANT: You must update Ansible .yml files when performing custom installations. These files should be updated with great caution. The syntax of Ansible files does not support using tabs for editing, only spaces. See the Ansible Documentation for more information and details about Ansible syntax.

Installation prerequisites

Before installing HPE Cray Programming Environment on HPE Cray Supercomputing EX systems running HPE Cray System Management (CSM), make sure that your system complies with supported systems. See Release information section of this guide for more details. Also, you must retain:

  • Root administrator access permissions to properly run the CPE Installer. Ansible needs these permissions to create the directory structure and install various elements of the CPE. Root access is not required to run the CPE; root access is required to only install or upgrade CPE.

  • Familiarity with:

    • Linux - To properly run the CPE Installer, an understanding of Linux file system basics is necessary.

    • Ansible - Knowledge of running Ansible and using Ansible playbooks is required. See Ansible Documentation for more information.

    • YAML - YAML is a human-readable data-serialization language. Ansible playbooks are stored in .yml format. Knowledge of YAML is not necessary to run Ansible playbooks but is useful for image customization.

    • Kubernetes (optional) - If you are installing CPE on containerized User Access Instance (UAI) nodes, an understanding of Kubernetes could be helpful but is not necessary to install or use the nodes.

Installing and upgrading CPE using the Install and Upgrade Framework (IUF) procedure

The Install and Upgrade Framework (IUF) is a CLI- and API-based process used to install, upgrade, and deploy CPE. The IUF process offers advantages for installing CPE onto compatible HPE Cray Supercomputing EX systems. These benefits include minimized user intervention, reduced time constraints, and a more automated and simplified method for installing CPE. The IUF method can be used with CSM 1.4 or higher.

The instructions in this chapter provides detailed steps and information for using the IUF process to install or upgrade CPE.

Installing or Upgrading CPE with IUF

This procedure details how to install or upgrade the base HPE Cray Programming Environment on an HPE Cray Supercomputing EX system using IUF. The same instructions are followed whether installing CPE using IUF for the first time or upgrading CPE using IUF on a previously installed system.

Prerequisite

Be sure to:

  • Review Installation Prerequisites before proceeding with these installation and upgrade procedures.

  • Download third-party compilers from their respective websites (for example, AOCC, Intel). CPE does not distribute third-party compilers.

  • Use CSM 1.4 or higher for installing or upgrading CPE.

Procedure

  1. SSH into the management node:

    user@hostname> ssh root@<system>-ncn-m001

  2. Create a directory for the activity media. For example:

    ncn-m001# mkdir -p /etc/cray/upgrade/csm/<activity_name>

  3. Copy media to your activity directory:

    ncn-m001# cd /etc/cray/upgrade/csm/<activity_name>
ncn-m001# cp ../reference_media/cpe-<CPE_RELEASE>-sles15-<spX>-\
csm-<CPE_VERSION>.tar.gz .

  4. Copy in reference bootprep config files:

    ncn-m001# cp -a /etc/cray/upgrade/csm/hpc-csm-software-recipe-23.1.18/vcs/* .

  5. If this is a CPE upgrade only, make a copy of /etc/cray/upgrade/admin/site_vars.yaml, and update the suffix and note values so that any artifacts created can be easily associated with the CPE upgrade.

    Example site_var.yaml:

    default:
   network_type: "cassini"
   suffix: "cpe-23.5.4.upgrade"
   note: "bob-"

  6. Run the iuf CLI command:

    ncn-m001# iuf -a cpemedia -m /etc/cray/upgrade/csm/cpemedia run --site-vars \
/etc/cray/upgrade/csm/site_vars.yaml --recipe-vars product_vars.yaml \
--bootprep-config-managed \
bootprep/compute-and-uan-bootprep.yaml --bootprep-config-management \
bootprep/management-bootprep.yaml -b process-media -e update-vcs-config

    The above example uses cpemedia as the activity_name.

    See the IUF section of the HPE Cray System Management Administration Guide for details on iuf command line options.

  7. Verify that CPE installed successfully. Use kubectl to print all CPE versions in the product catalog configmap, and double-check that the latest version is also in the list. Note that the latest CPE version is likely not at the end of the output; scroll up through the output to locate the latest CPE version.

    ncn-m001# kubectl get cm cray-product-catalog -n services -o json | jq -r .data.cpe
...
<CPE_VERSION>:
  configuration:
    clone_url: https://vcs.hostname.com/vcs/cray/cpe-config-management.git
    commit: 341017e953c3c57dd46ddbccec168ca28af9199a
    import_branch: cray/cpe/<CPE_VERSION>
    import_date: 2023-01-24 20:10:42.950742
    ssh_url: git@host.com:cray/cpe-config-management.git

  8. (Optional) Go the Creating and Deploying a Custom Image using Release Templates to build CPE images locally, instead of using pre-built CPE images. Building CPE images locally offers more flexibility with managing installation images.

  9. (Optional) Upload third-party artifact(s) to a Nexus repository.

    Change directory to the expanded CPE artifacts that exist in the media folder. In this case, use /etc/cray/upgrade/csm/cpemedia. Third-party packages can be copied over to the /etc/cray/upgrade/csm/cpemedia folder.

    ncn-m001# cd /etc/cray/upgrade/csm/cpemedia/cpe-<CPE_VERSION>-sles15-sp4

    The CPE release tar file contains a script, install-3p.sh, that uploads third-party packages to Nexus repositories. New repositories are automatically created if they do not already exist. The script has two modes of operation:

    • Uploading a

      ncn-m001# install-3p.sh <FILE> <REPO_NAME>

    • Uploading RPM files, where <RPM_DIR> is a directory of RPMs.

      ncn-m001# install-3p.sh <RPM_DIR> <REPO_NAME>

    Uploading RPM files automatically generates RPM repository metadata required for installation using Zypper. For example:

    AMD AOCC Compiler

    ncn-m001# cpe-<CPE_RELEASE>-sles15-<spX>/install-3p.sh \
aocc-compiler-3.2.0.tar aocc-compiler-3.2.0-linux-x86_64-raw

    Linaro Forge

    ncn-m001# cpe-<CPE_RELEASE>-sles15-<spX>/install-3p.sh \
linaro-forge-<VERSION>-linux-<ARCH>.tar linaro-forge-<VERSION>-linux-<ARCH>-raw

    Intel oneAPI

    Note that the oneAPI 2022.2.0 release uses the version string “2022.2.0” for RPM versions and installation paths. Therefore, it is the version number needed for installation scripts.

    ncn-m001# tar xf intel-oneapi-2022.2.0.tar
ncn-m001# cpe-<CPE_RELEASE>-sles15-sp4/install-3p.sh \
intel-oneapi-2022.2.0/ intel-oneapi-2022.2.0

    Perforce TotalView

    ncn-m001# cpe-<CPE_RELEASE>-sles15-<spX>/install-3p.sh \
totalview-<VERSION>.<ARCH>.tar totalview-<VERSION>-linux-<ARCH>-raw

Installation of CPE is now complete. If other HPE Cray Supercomputing EX software products are being installed or upgraded in conjunction with CPE, refer to the HPE Cray Supercomputing EX System Software Stack Installation and Upgrade Guide for CSM (S-8052) to determine which step to execute next; see HPE Cray Supercomputing EX software documentation links for a direct link. Otherwise, continue to the next sections of this document for operations to configure and deploy new CPE images.

Optional third-party product image customization

This section provides instructions for configuring third-party compilers (such as AOCC, Linaro Forge, and Intel oneAPI) into a new CPE image for CPE deployment.

Prerequisites

The CPE package must be installed, and third-party artifacts must be available in a Nexus repository.

Procedure

While HPE provides Ansible customization roles for the AMD AOCC Compiler, Intel oneAPI, and Linaro Forge, some steps detailed in this procedure use the AOCC customization as an example; however, the procedure is similar for other products. Product ansible roles include:

  • AMD AOCC Compiler: cray.pe_aocc_customize

  • Intel oneAPI: cray.pe_intel_customize

  • Linaro Forge: cray.pe_forge_customize

  • Perforce TotalView: cray.pe_totalview_customize

  • NVIDIA: cray.pe_nvidia_customize

For products supporting GPUs (NVIDIA HPC SDK, AMD ROCm), refer to the GPU Support section of HPE Cray Supercomputing User Services Software Administration Guide: CSM on HPE Cray Supercomputing EX Systems for installation instructions; see HPE Cray Supercomputing EX software documentation links for a direct link to the appropriate guide.

  1. (IUF Installation) Clone and create the local VCS repository:

    ncn-m001# mkdir -p /var/tmp/cpe
ncn-m001# cd /var/tmp/cpe
ncn-m001# git clone https://crayvcs@api-gw-service-nmn.local/vcs/cray/cpe-config-management.git
Enter password: <password obtained from secret credentials>
ncn-m001# cd /var/tmp/cpe/cpe-config-management
ncn-m001# git checkout integration-<CPE_RELEASE>

  2. (Non-IUF Installation) The CPE install.sh script executed earlier cloned a local VCS repository. Change directory into the new path and continue:

    ncn-m001# cd /var/tmp/cpe/cpe-config-management

  3. Verify that the default variables for the image customization role match the values used earlier for uploading to Nexus:

    ncn-m001# vi roles/cray.pe_aocc_customize/defaults/main.yml

    For the cray.pe_intel_customize role, intel_pkgs can be modified to install a different set of oneAPI components.

  4. (Linaro Forge Only) Copy the license file to roles/cray.pe_forge_customization/files/License.dat or populate the existing empty License.dat file with the license information.

  5. (Perforce Totalview only) Copy the License.dat or tv_license_file license file to roles/cray.pe_totalview/files/, and update roles/cray.pe_totalview/defaults/main.yml based on the type of license.

    Example using the FNP license:

    totalview:
...
license_path: "/opt/toolworks/FNP_license"
license_ "License.dat"

    Example using the FNE license:

    totalview:
...  
license_path: "/opt/toolworks/FNE_license"
license_ "tv_license_file"

  6. Add, commit, and push changes:

    ncn-m001# git commit -am "Add customizations to install the AMD AOCC compiler"
ncn-m001# git push -u origin cpe-<CPE_RELEASE>-integration

  7. Run the CPE image customization script with a parameter (aocc, intel, nvidia, forge, totalview), specifying which built-in playbook to use:

    If a previous version of CPE is installed:

    a. Determine the IMS image ID of the cpe-barebones image:

    ncn-m001# kubectl get cm cray-product-catalog -n services -o json | jq -r .data.cpe

    b. Set the environment variable BASE_IMG_ID to the IMS image ID:

    ncn-m001# export BASE_IMG_ID=<IMS_image_ID>

    c. Run the CPE image customization script:

    ncn-m001# ./cpe-custom-img.sh aocc

    A new image (deployable with the provided CPE image) is created after the CFS session completes.

  8. Record the result_id (img_id) for use when preparing the CPE deployment:

    ncn-m001# cray cfs sessions describe cpe-aocc-customization \
--format json | jq -r .status.artifacts[].result_id

0e54050a-c43c-4534-ba38-7191838e348d

  9. Record the img_etag for use when preparing the CPE deployment:

    ncn-m001# cray artifacts describe boot-images <img_id>/rootfs --format json | \
jq -r .artifact.ETag

  10. Set the sbps-project metadata key to true:

    ncn-m001# cray ims images update <img_id> --metadata-operation set \
--metadata-key sbps-project --metadata-value true

Repeat the steps above for each third-party product image that needs CPE support customization. Then, continue to the Configuring CPE Using CFS section to prepare the CPE deployment.

Creating and deploying a custom image using release templates

The information in this sections provides instructions for building and deploying a custom CPE image that is based on an official release manifest and any extra functionality not included in the official release. The installation method described in this chapter is an alternative method to the traditional installation method, which uses pre-built CPE images. It offers more flexibility over what content goes into CPE images.

Procedure

  1. Clone the CPE local VCS repository, and switch to the latest integration branch:

    ncn-m001# mkdir -p /var/tmp/cpe
ncn-m001# cd /var/tmp/cpe
ncn-m001# export gp=$(kubectl get secret -n services vcs-user-credentials --template={{.data.vcs_password}} | base64 --decode)
ncn-m001# git clone https://crayvcs:"${gp}"@api-gw-service-nmn.local/vcs/cray/cpe-config-management.git
ncn-m001# cd /var/tmp/cpe/cpe-config-management
ncn-m001# git checkout integration-<CPE_RELEASE>

  2. Update the default variables file with CPE repository and RPMs to install. This file is populated with latest CPE release RPMs and can be customized by adding or deleting components. Specifically:

    a. Update the cpe_<name>_pkgs lists with all of the RPMs to install.

    b. (Optional for installing RPMs not included in a CPE release) Uncomment the cpe_extra_repo section, and update the url and name fields.

    c. (Optional for installing RPMs not included in a CPE release) Fill in the cpe_extra_pkgs list with additional RPMs not from a CPE release:

    ncn-m001# vi roles/cray.pe_base_customize/defaults/main.yml

cpe_cce_enabled: true
cpe_csml_enabled: true
cpe_mpt_enabled: true
pe_tools_enabled: true

### Additional PrgEnvs
prgenv_amd_enabled: false
prgenv_nvidia_enabled: false
prgenv_intel_enabled: false
prgenv_aocc_enabled: false
...
################################################################################
### Uncomment & update for additional repositories to install pkgs from
#cpe_extra_repo:
#  url: "{{ nexus_repo_url }}/cpe-25.03-extra"
#  name: "cpe-25.03-extra"
################################################################################
### List of packages to install
cpe_base_pkgs: []
cpe_extra_pkgs: []
cpe_cce_pkgs: []
cpe_mpt_pkgs: []

  3. (Optional step for installing RPMs not included in a CPE release) Create an RPM_LIST_FILE, which comprises additional package names, and afterwards, use the fetch2nexus.sh script.

    RPM_LIST_FILE=cpe-25.03-extra-sles15-sp6-recipe.rpmlist

cat > $RPM_LIST_FILE <<EOF
atp-3.15.5-20241007204318_cb3a010cbbd6-140.sles15sp5.x86_64.rpm
cce-18.0.1-202410021623.ac9591613de6f-0.sles15sp5.x86_64.rpm
cce-18.0.1-runtime-202410021623.ac9591613de6f-0.sles15sp5.x86_64.rpm
...
EOF

  4. (Optional step for installing RPMS not included in a CPE release) Run the fetch2nexus.sh script to download RPMs from a source URL, then upload it to local Nexus. See the script comments for details on parameter expectations. For example:

    ./fetch2nexus.sh cpe-25.03-extra \
https://<usr>:<token>@upd.linux.hpe.com/repo/cpe/25.03/sles/sles15sp6/x86_64 \
$RPM_LIST_FILE

  5. Add, commit, and push changes:

    ncn-m001# git commit -am "Add customizations to install the CPE base image"
ncn-m001# git push -u origin integration-<CPE_RELEASE>

  6. Run the CPE image customization script:

    a. Determine the IMS image ID of the cpe-barebones image:

    ncn-m001# kubectl get cm cray-product-catalog -n services -o json | \ 
jq -r .data.cpe

    b. Set the BASE_IMG_ID environment variable to the IMS image ID:

    ncn-m001# export BASE_IMG_ID=<IMS_image_ID>

    c. Run the CPE image customization script with the base parameter:

    ncn-m001# ./cpe-custom-img.sh base

    A new deployable image is created after the CFS session completes.

    d. Record the image ID and etag from the script output to be used for updating the pe_deploy.yml playbook.

    e. Follow the steps as documented in the Configuring CPE using CFS, except use the new customized base image as if it were a third-party image (such as Intel, AOCC, or NVIDIA).

Configuring CPE using CFS

This chapter provides details for preparing a CPE CFS layer for product integration.

Prerequisites

Make sure you have:

Procedure

Note: Some of the steps in this procedure are specific to IUF installations. Steps annotated with Non-IUF Installation are for non-IUF environments. Steps annotated with IUF Installation are specific only to IUF environments.

  1. Clone the CPE configuration management repository, and checkout the integration branch. The CPE install.sh or iuf run command executed earlier created a new local integration branch.

    ncn-m001# mkdir -p /var/tmp/cpe
ncn-m001# cd /var/tmp/cpe
ncn-m001# git clone https://crayvcs@api-gw-service-nmn.local/vcs/cray/cpe-config-management.git
Enter password: <password obtained from secret credentials>
ncn-m001# cd /var/tmp/cpe/cpe-config-management
ncn-m001# git checkout integration-<CPE_RELEASE>

  2. Configure images to deploy. The order of roles is important: The first is the top-most layer and also the default image; lower layers and non-default images must follow.

    The cray_pe_pkg parameter values include:

    • base: Contains the base CPE content, including PrgEnv-cray and PrgEnv-gnu. This value is the default value if cray_pe_pkg is not set.

    • intel: Contains PrgEnv-intel for Intel oneAPI support.

    • aocc: Contains PrgEnv-aocc for AMD Optimizing C/C++ Compiler support.

    • amd: Contains PrgEnv-amd for AMD ROCm support. Deploy this image for AMD GPU-enabled systems.

    • nvidia: Contains PrgEnv-nvhpc for NVIDIA HPC SDK support. Deploy this image for NVIDIA GPU-enabled systems.

    To deploy a customized image, set the img_etag (required for SBPS in CSM v1.6 and later) and img_id to the IMS image etag and ID of the customized image (see the Optional third-party product image customization section), and give the image a unique name with img_name. After the job completes, retrieve the img_id and img_etag, and add them to the pe_deploy.yml file. As an example, to retrieve an img_etag, you could enter:

    cray cfs sessions describe cpe-intel-customization --format json | jq -r \
.status.artifacts[].result_id

    Note that:

    • If you are deploying the customized image using DVS, avoid using the IMS etag for the image, and

    • The etag required for SBPS is not the same as the etag required for traditional DVS usage.

    The following is an example of how to deploy CPE base and aocc images with the current and a previous version of CPE, along with two versions of the AOCC compiler.

    Example:

    Deploying supported third-party CPE images for both x86 and ARM/AArch64 nodes, in addition to hybrid environments:

      ncn-m001# vim pe_deploy.yml
  roles:
    - { role: cray.pe_deploy, when: not cray_cfs_image }
    - { role: cray.pe_deploy, cray_pe_version: "21.10", when: not cray_cfs_image }
    - { role: cray.pe_deploy, cray_pe_pkg: aocc, when: (not cray_cfs_image) and \
        (ansible_architecture == 'x86_64') }
    - { role: cray.pe_deploy, cray_pe_pkg: aocc, cray_pe_version: "21.10", when: \
      (not cray_cfs_image) and (ansible_architecture == 'x86_64') }
    - { role: cray.pe_deploy, img_name: "aocc-compiler-3.1.0", \
        img_id: "1f506586-e447-4c2a-b38d-1158cb29e4f8", 
        img_etag: "8344721a5a376b8924fe2afc44a13968-79", 
        when: (not cray_cfs_image) and \
        (ansible_architecture == 'x86_64') }
    - { role: cray.pe_deploy, img_name: "aocc-compiler-3.0.0", \
        img_id: "0e54050a-c43c-4534-ba38-7191838e348d", 
        img_etag: "8372929446288292ab3428ae82b22881-34", 
        when: (not cray_cfs_image) and \
        (ansible_architecture == 'x86_64') }

    If the ansible_architecture variable is undefined, the system automatically determines x86_64 or AArch64 nodes on which to deploy, such as the first two example lines above (base-latest and base-21.10). If the variable is defined, the update applies to the target set of nodes. In the above example, aocc-latest, aocc-21.10, aocc-compiler-3.1.0, and aocc-compiler-3.0.0 are limited to x86_64 nodes since they are not supported on AArch64.

    You can use:

    • git diff origin/integration-<prev_release>.. to check differences between latest and previous integration branches.

    • git checkout origin/integration-<prev_release> – pe_deploy.yml to pick up previously customized files, as needed.

  3. (Optional) Customize site modules. You can set in roles/cray.pe_deploy/default/main.yml custom values to meet site-specific needs for:

    • cray-pe-configuration.csh.j2

    • cray-pe-configuration.sh.j2

    The above modules are in roles/cray.pe_deploy/templates.

    Acceptable variables include:

    • cray_pe_module_prog: Defines the default module handling system, either Lmod (Lua) or Environment Modules (TCL).

    • cray_pe_default_prgenv: Defines the default programming environment.

    • cray_pe_mpaths: Defines any site-specific paths to be added to MODULEPATH to make site modules available.

    • cray_pe_init_module_list: Defines the modules to be loaded on login.

    • cray_pe_site_module_list: Defines additional site modules to be loaded upon login.

    • cray_pe_prgenv_module_list: Defines modules to be swapped as part of the PrgEnv module.

    • cray_pe_one_off_set_defaults: Defines a list of paths to set_default scripts to be run at deploy time. This variable enables you to set default versions at the component level.

    For example, to set Lmod as the default module handling system in the image in roles/cray.pe_deploy/defaults/main.yml, set cray_pe_module_prog: lmod.

  4. Commit and push the changed files in git:

    ncn-m001# git commit -am "Update CPE packages and image layers"
ncn-m001# git push -u origin integration-<CPE_RELEASE>

  5. (IUF installation) Observe that the IUF creates the CFS configuration layer for CPE, provides a bootprep file that is used to customize images/personalize nodes, and creates BOS session templates.

    Example:

    The example specifies activity_name == cpemedia and shows other example paths. The administrator sets up IUF-related files in Installing and upgrading CPE using the Install and Upgrade Framework (IUF) procedure.

    Note: On systems with only AArch64 nodes:

    a. Edit the /etc/cray/upgrade/csm/site_vars.yaml, and using the latest CPE version, for example, add:

    cpe-aarch64:
  version: 23.12.3
  working_branch: "{{ working_branch }}"

    b. Edit both the /etc/cray/upgrade/csm/cpemedia/bootprep/compute-and-uan-bootprep.yaml file and the management-bootprep.yaml file by updating them from:

    name: cpe
version: "{{cpe.version}}"
branch: "{{cpe.working_branch}}"

    To:

    name: cpe-aarch64
version: "{{cpe_aarch64.version}}"
branch: "{{cpe_aarch64.working_branch}}"

    ncn-m001# cd /etc/cray/upgrade/csm/cpemedia
ncn-m001# iuf -a cpemedia -m /etc/cray/upgrade/csm/cpemedia run --site-vars \
  /etc/cray/upgrade/csm/site_vars.yaml --recipe-vars product_vars.yaml \
  --bootprep-config-managed bootprep/compute-and-uan-bootprep.yaml \
  --bootprep-config-management bootprep/management-bootprep.yaml \
  -b update-cfs-config -e prepare-images

  6. (Non-IUF installation) CPE includes an operation automation script that creates a new CFS configuration with the latest CPE version and commit ID. The script has two optional parameters (CFS_name and apply):

    ncn-m001# cpe-cfs.sh [CFS_name] [apply]

    Note, however, that if:

    • No parameters are specified; the script uses the latest cpe-yy.mm-integration branch in CFS.

    • SAT (excluding SAT 2.2.15 or earlier) is installed; the script also outputs a section of yaml code for use in a sat bootprep input file for integration with other products. Refer to the SAT bootprep section of the SAT product stream documentation for more information. See HPE Cray Supercomputing EX software documentation links for a direct link.

    • The CFS_name parameter is specified; the script proposes a new .json file that adds or replaces any existing CPE layer.

    • The apply parameter is specified; the script modifies the CFS config using the proposed .json file. HPE recommends a trial run without the apply parameter to verify the results, then rerun with the apply parameter to incorporate the changes. For example:

      ncn-m001# ./cpe-cfs.sh cos-config-2.1.27 [apply]
...
  ________________________________________
  Updating new CPE CFS configuration ...
  {
    "lastUpdated": "2021-10-13T21:05:40Z",
    "layers": [
      {
        "cloneUrl": "https://api-gw-service-nmn.local/vcs/cray/cpe-config-management.git",
        "commit": "4194bd87979f876400fa9159a60985dacee06a3b",
        "name": "cpe-21.11-integration",
        "playbook": "pe_deploy.yml"
      }
    ],
    "name": "cpe-21.11-integration"
  }
  ________________________________________
  Generating new layers for cos-config-2.0.27 ...
  Proposed new layers for cos-config-2.0.27:
  {
    "layers": }
...

  7. (Non-IUF installation) If sat bootprep is not used (for example, for NCN-personalization of UAI hosts), then the cpe-cfs.sh [CFS_name] parameter must be specified:

    a. (HPE recommended) Run the script without the apply parameter to verify the results, then rerun it again with the apply parameter to incorporate the changes. Do not use the apply parameter if sat bootprep is used.

    b. Rerun the script with the apply parameter for COS, UAN, and NCN personalization CFS configurations, as necessary.

    c. Update BOS session templates to ensure the latest CPE CFS configs are included on all nodes after reboots. Refer to Configuration Management in the Cray System Management Administration Guide for details; see HPE Cray Supercomputing EX software documentation links for a direct link.

    d. Run module list to check if PE is ready to use after CFS completes on a compute or UAN node.

    Example Output:

    Note that module versions below are examples only, and may differ from those currently loaded on the system. For current CPE release product versions, see the release announcement.

    nid000001# module list
Currently Loaded Modulefiles:
1) craype-x86-rome          5) xpmem/2.6.2-2.5_2.27--gd067c3f.shasta   9) cray-mpich/8.1.28
2) libfabric/1.15.2.0       6) cce/17.0.0                             10) cray-libsci/23.12.5
3) perftools-base/23.12.0   8) craype/2.7.30                          11) PrgEnv-cray/8.5.0
4) craype-network-ofi       7) cray-dsmml/0.2.2

Installing previously released CPE packages for CSM on HPE Cray Supercomputing EX systems

This chapter provides instructions for installing a previously released CPE package, <PREV_RELEASE>, after installing the latest CPE.

Prerequisites

Note that:

  • HPE CPE must be installed on an HPE Cray Supercomputing EX system running CSM. For version requirements, see Installation Prerequisites.

  • Previously released CPE packages must use the installer that comes with the latest CPE package.

Procedure

This procedure installs the <PREV_RELEASE> package along with the latest release.

Important: Throughout this procedure, replace instances of <PREV_RELEASE> with the desired previous release YY.MM value.

  1. Download the old CPE .tar file, extract it into a path <untar_path>, and then run:

    ncn-m001# SQFS=CPE-base.x86_64-<PREV_RELEASE>.squashfs
ncn-m001# cray artifacts create boot-images PE/$SQFS <untar_path>/squashfs/$SQFS

  2. Check out the integration branch from the VCS git repo, and update two files to include the <PREV_RELEASE> package for deployment:

    ncn-m001# git checkout integration-<PREV_RELEASE>
ncn-m001# vi pe_deploy.yml
roles:
    - { role: cray.pe_deploy, when: not cray_cfs_image }
    - { role: cray.pe_deploy, cray_pe_version: <PREV_RELEASE>, when: not cray_cfs_image }

  3. Update the CFS configuration (compute/COS, management-x.y, and UAN) to point to the new integration branch commit ID.

About the Lmod Custom Dynamic Hierarchy

Lmod enables a user to dynamically modify their user environment through Lua modules. CPE capitalizes on the Lmod hierarchical structure, including the Lmod module auto-swapping functionality. This functionality allows module dependencies to determine the branches of the tree-like hierarchy. Lmod allows static and dynamic hierarchical module paths. Lmod provides full support for static paths, which build the hierarchy based on the current set of modules loaded. Alongside static paths, CPE implements dynamic paths for a subset of the Lmod hierarchy (compilers, networks, CPUs, and MPIs). Dynamic paths give an advanced level of flexibility for detecting multiple dependency paths and allow custom paths to join the existing CPE Lmod hierarchy without modifying customer modulefiles.

Static Lmod hierarchy

Modules dependent on one or more modules being loaded are not visible to a user until their prerequisite modules are loaded. When the prerequisite modules are loaded, it adds the static paths of the dependent modules to the MODULEPATH environment variable, thereby exposing the dependent modules to the user. For more detailed information on Lmod static module hierarchies, please consult User Guide for Lmod.

Dynamic Lmod hierarchy

The CPE custom dynamic Lmod hierarchy abbreviates the overall Lmod hierarchy tree by relying on compatibility and not directly on a prerequisite version. Therefore, dependent modules do not need to exist in a new branch every time their prerequisite modules change versions. Instead, dynamic paths use a compatibility version that increases when a new prerequisite module version breaks compatibility in some way. The number following the module path alias (for example, 1.0 in x86-rome/1.0 and ofi/1.0) identifies the compatible version.

Module path aliases and current compatibility versions

Compatible versions listed in the following tables include the minimum supported versions.

Compiler

SLES Module Alias/Compatible Version

amd

amd/7.0

cce

crayclang/21.0

gcc

gnu/13.0

aocc

aocc/4.1

intel

intel/2023.2

nvidia (x86)

nvidia/20

Network

Module Alias/Compatible Version

craype-network-none

none/1.0

craype-network-ofi

ofi/1.0

craype-network-ucx

ucx/1.0

CPU

Module Alias/Compatible Version

craype-x86-milan

x86-milan/1.0

craype-x86-rome

x86-rome/1.0

craype-x86-trento

x86-trento/1.0

MPI

Module Alias/Compatible Version

cray-mpich

cray-mpich/8.0

cray-mpich-abi

cray-mpich/8.0

cray-mpich-abi-pre-intel-5.0

cray-mpich/8.0

cray-mpich-ucx

cray-mpich/8.0

cray-mpich-ucx-abi

cray-mpich/8.0

cray-mpich-ucx-abi-pre-intel-5.0

cray-mpich/8.0

Custom dynamic hierarchy

The CPE custom dynamic hierarchy extension allows custom module paths to join the existing CPE Lmod hierarchy implementation without modifying customer modulefiles. The custom dynamic module types that CPE supports include:

  • Compiler

  • Network

  • CPU

  • MPI

  • Compiler/Network

  • Compiler/CPU

  • Compiler/Network/CPU/MPI

As each custom dynamic module type loads, a handshake occurs using special pre-defined environment variables. When all hierarchical prerequisites are met, the paths of the dependent modulefiles are added to the MODULEPATH environment variable, thereby exposing the dependent modules to the user.

For Lmod to assist a user optimally, load the compiler, network, CPU, and MPI module. Lmod cannot detect modules hidden in dynamic paths without one of each type of module being loaded.

Creating a custom dynamic hierarchy

For the CPE custom dynamic hierarchy to detect the desired Lmod module path, one or more custom dynamic environment variables must be created according to the requirements defined within this procedure.

Prerequisites

Set Lmod as the default module handling system before initiating this procedure.

Procedure

To create a custom dynamic environment variable:

  1. Begin the environment variable name with LMOD_CUSTOM_.

  2. Append the descriptor of the module type that the environment variable will represent. The module types and descriptors are:

    Module Type

    Descriptor

    Compiler

    COMPILER_

    Network

    NETWORK_

    CPU

    CPU_

    MPI

    MPI_

    Compiler/Network

    COMNET_

    Compiler/CPU

    COMCPU_

    Compiler/Network/CPU/MPI

    CNCM_

    Example: The custom dynamic environment variable for the combined compiler and CPU module begins with LMOD_CUSTOM_COMCPU_.

  3. Following the descriptor, append all prerequisite module aliases along with their respective compatible versions. See the Module Path Aliases and Current Compatibility Versions section of this guide for more information. The format of the module path alias/compatible version string for each module type is shown below. Note that due to publishing issues, long module alias/compatible version strings are split across two lines as indicated below.

    Module Type: Module Path Alias/Compatible Version String

    Compiler: <compiler_name>/<compatible_version>

    Network: <network_name>/<compatible_version>

    CPU: <cpu_name>/<compatible_version>

    MPI:

    <compiler_name>/<compatible_version>/<network_name>/<compatible_version>/

    <mpi_name>/<compatible_version>

    Compiler/Network: <compiler_name>/<compatible_version/<network_name>/<compatible_version>

    Compiler/CPU: <compiler_name>/<compatible_version>/<cpu_name>/<compatible_version>

    Compiler/Network/CPU/MPI:

    <compiler_name>/<compatible_version>/<network_name>/<compatible_version>/

    <cpu_name>/<compatible_version>/<mpi_name>/<compatible_version>

    To create an acceptably formatted environment variable name, replace all slashes and dots in the module alias/compatible version string with underscores. Also, all letters must be in uppercase format.

    Example Module Path Alias/Compatible Version Strings:

    • Compiler = cce

      The path alias/compatible version string (values found in Module Path Aliases and Current Compatibility Versions section of this guide) is crayclang/10.0; therefore, the text added to the environment variable name is:

      CRAYCLANG_10_0

    • Network = craype-network-ofi

      The path alias/compatible version string is ofi/1.0; therefore, the environment variable text is:

      OFI_1_0

    • CPU = craype-x86-rome

      The path alias/compatible version string is x86-rome/1.0; therefore, the environment variable text is:

      X86_ROME_1_0

    • MPI = cray-mpich

      cray-mpich has two prerequisite module types (compiler and network). Therefore, the environment variable must include the alias/compatible version for the desired compiler, network, and MPI. For a cray-mpich module dependent on cce and craype-network-ofi, the path alias/compatible version string is crayclang/10.0/ofi/1.0/cray_mpich/8.0; therefore, the environment variable text is:

      CRAYCLANG_10_0_OFI_1_0_CRAY_MPICH_8_0.

    • Compiler/Network = cce with craype-network-ofi

      The path alias/compatible version string is crayclang/10.0/ofi/1.0; therefore, the environment variable text is:

      CRAYCLANG_10_0_OFI_1_0

    • Compiler/CPU = cce with craype-x86-rome

      The path alias/compatible version string is crayclang/10.0/x86-rome/1.0; therefore, the environment variable text is:

      CRAYCLANG_10_0_X86_ROME_1_0

    • Compiler/Network/CPU/MPI = cce, craype-network-ofi, craype-x86-rome, and cray-mpich

      The path alias/compatible version string is crayclang/10.0/ofi/1.0/x86-rome/1.0/cray-mpich/8.0; therefore, the environment variable text is:

      CRAYCLANG_10_0_OFI_1_0_X86_ROME_1_0_CRAY_MPICH_8_0

  4. Append _PREFIX following the final module/compatibility text instance:

    Example: Network = craype-network-ofi

    The custom dynamic environment variable is LMOD_CUSTOM_NETWORK_OFI_1_0_PREFIX.

    Creation of the custom dynamic environment variable is now complete.

  5. Add the custom dynamic environment variable to the user environment by exporting it with its value set to the Lmod module path:

    # export LMOD_CUSTOM_NETWORK_OFI_1_0_PREFIX=<lmod_module_path>

    Example: Network = craype-network-ofi

    All modulefiles in <lmod_module_path> are shown to users whenever craype-network-ofi is loaded.

Support matrices for previous releases

This chapter lists CPE-supported components, third-party software, and modules supported for applicable and previous releases of the CPE software. This information is provided for reference purposes.

CPE release matrices for SLES

CPE supports various SLES-based software components, including SLES for Aarch64 and x86 architectures. These components include compilers, libraries, debugging/profiling tools, programming models and so forth. Supported version of these components are updated with each release of CPE. This section lists which SLES-based component versions are supported for each CPE release.

SLES AArch64 support matrix

SLES with AArch64 systems is supported with CPE on HPE Cray Supercomputing EX systems with either CSM or HPCM. Below are product components, modules, third-party software versions supported with previous CPE releases with these configurations.

(D) represents the default version installed at installation.

* HPCM only

Release

CPE 25.09

CPE 25.09

CPE 25.03

CPE 25.03

24.11

24.11

24.07

Product

sles15sp7-aarch64 *

sles15sp6-aarch64

sles15sp6-aarch64

sles15sp5-aarch64

sles15sp5-aarch64

sles15sp6-aarch64

sles15sp5-aarch64

COS

25.9

25.9

25.3

24.7

25.1

24.7

24.7

COS Base

N/A

N/A

3.3.0

3.1.0

3.2.0

3.1.0

3.1.0

CSM

Not supported

1.7

1.6.1

1.6.1

1.6

1.6

1.5

HPCM

1.14

1.14

1.13

1.13

1.12

1.12

1.11

USS

1.4.0

1.4.0

1.3.0

1.1.0

1.2.0

1.1.0

1.1.0

amd

aocc

5.0

5.0

4.2

4.2

4.2**

atp

3.15.7 (D)

3.15.7 (D)

3.15.6 (D)

3.15.6 (D)

3.15.5 (D)

3.15.5 (D)

3.15.4 (D)

cce

20.0.0

20.0.0

19.0.0 (D)

19.0.0 (D)

18.0.1 (D)

18.0.1 (D)

18.0.0 (D)

cpe-gcc-mpfr

3.1.4 (D)

3.1.4 (D)

3.1.4 (D)

3.1.4 (D)

3.1.4 (D)

3.1.4 (D)

cpe-gcc-native

14.2 (D)

14.2 (D)

cpe-gcc-native

14 (D)

14 (D)

13.2

13.2

13.2 (D)

13.2 (D)

13.2 (D)

cpe-gcc-native

13

12.3

12.3

12.3

12.3

12.3

cpe-gcc-native

12

cpe-prgenv-amd

cpe-prgenv-aocc

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-cray

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

cpe-prgenv-cray-amd

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-gnu

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

cpe-prgenv-gnu-amd

cpe-prgenv-intel

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-nvhpc

8.5.0 (D)

cpe-prgenv-nvidia

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

cray-R

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

cray-ccdb

5.0.7 (D)

5.0.7 (D)

5.0.6 (D)

5.0.6 (D)

5.0.5 (D)

5.0.5 (D)

5.0.4 (D)

cray-cdst-support

2.14.6 (D)

2.14.6 (D)

2.14.5 (D)

2.14.5 (D)

2.14.3 (D)

cray-cti

2.20.0 (D)

2.20.0 (D)

2.19.1 (D)

2.19.1 (D)

2.19.0 (D)

2.19.0 (D)

2.18.4 (D)

cray-dsmml

0.3.1 (D)

0.3.1 (D)

0.3.1 (D)

0.3.0 (D)

0.3.0 (D)

0.3.0 (D)

0.3.0 (D)

cray-dwarf

2.0.0 (D)

2.0.0 (D)

0.11.1 (D)

0.11.1 (D)

0.11.0 (D)

0.11.0 (D)

0.9.2 (D)

cray-dyninst

12.3.6 (D)

12.3.6 (D)

12.3.5 (D)

12.3.5 (D)

12.3.4 (D)

12.3.4 (D)

12.3.2 (D)

cray-fftw

3.3.10.11 (D)

3.3.10.11 (D)

3.3.10.10 (D)

3.3.10.10 (D)

3.3.10.9 (D)

3.3.10.9 (D)

3.3.10.8 (D)

cray-hdf5

1.14.3.7 (D)

1.14.3.7 (D)

1.14.3.5 (D)

1.14.3.5 (D)

1.14.3.3 (D)

1.14.3.3 (D)

1.14.3.1 (D)

cray-libsci

25.09.0 (D)

25.09.0 (D)

25.03.0 (D)

25.03.0 (D)

24.11.0 (D)

24.11.0 (D)

24.07.0 (D)

cray-libsci-acc

25.09.0 (D)

25.03.0 (D)

25.03.0 (D)

24.11.0 (D)

24.11.0 (D)

24.07.0 (D)

cray-lmod

8.7.60 (D)

8.7.60 (D)

8.7.55 (D)

8.7.55 (D)

8.7.37 (D)

8.7.37 (D)

8.7.37 (D)

cray-modules

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

cray-mpich

8.1.33

8.1.33 (D)

8.1.32 (D)

8.1.32 (D)

8.1.31 (D)

8.1.31 (D)

8.1.30 (D)

cray-mpich

9.0.1 (D)

9.0.1 (D)

9.0.0

9.0.0

cray-mpixlate

1.0.7 (D)

1.0.7 (D)

1.0.6 (D)

1.0.6 (D)

1.0.5 (D)

cray-mrnet

5.1.6 (D)

5.1.6 (D)

5.1.5 (D)

5.1.5 (D)

5.1.4 (D)

5.1.4 (D)

5.1.3 (D)

cray-netcdf

4.9.2.1 (D)

4.9.2.1 (D)

4.9.0.17 (D)

4.9.0.17 (D)

4.9.0.15 (D)

4.9.0.15 (D)

4.9.0.13 (D)

cray-open-shmemx

11.7.5 (D)

11.7.5 (D)

11.7.4 (D)

11.7.3 (D)

11.7.3 (D)

11.7.3 (D)

11.7.2 (D)

cray-papi

7.2.0.2 (D)

7.2.0.2 (D)

7.2.0.1 (D)

7.2.0.1 (D)

7.1.0.4 (D)

7.1.0.4 (D)

7.1.0.2 (D)

cray-parallel-netcdf

1.12.3.19 (D)

1.12.3.19 (D)

1.12.3.17 (D)

1.12.3.17 (D)

1.12.3.15 (D)

1.12.3.15 (D)

1.12.3.13 (D)

cray-pe-set-default

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

cray-pmi

6.1.16 (D)

6.1.16 (D)

6.1.15 (D)

6.1.15 (D)

6.1.15 (D)

6.1.15 (D)

6.1.15 (D)

cray-pmi-devel

6.1.16

6.1.16

6.1.15

6.1.15

6.1.15

6.1.15

6.1.15

cray-pmi-doc

6.1.16

6.1.16

6.1.15

6.1.15

6.1.15

6.1.15

6.1.15

cray-python

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

cray-stat

3.11.7 (D)

4.12.6 (D)

4.12.5 (D)

4.12.5 (D)

4.12.4 (D)

4.12.4 (D)

4.12.3 (D)

cray-ucx

cray-zmqnet

1.3.2 (D)

1.3.2 (D)

1.3.0 (D)

1.3.0 (D)

1.0.0 (D)

1.0.0 (D)

craype

2.7.35 (D)

2.7.35 (D)

2.7.34 (D)

2.7.34 (D)

2.7.33 (D)

2.7.33 (D)

2.7.32 (D)

craype-dl-plugin-ftr

craype-dl-plugin-py3

24.07.1 (D)

24.07.1 (D)

24.07.1 (D)

24.07.1 (D)

24.07.1 (D)

craype-targets-ex

1.16.0 (D)

1.16.0 (D)

1.15.1 (D)

1.15.1 (D)

1.15.0 (D)

1.15.0 (D)

1.13.2 (D)

craypkg-gen

1.3.36 (D)

1.3.36 (D)

1.3.35 (D)

1.3.35 (D)

1.3.34 (D)

1.3.34 (D)

1.3.33 (D)

forgesup

24.1.1

24.1.1

23.1.2

23.1.2

23.1.2

gdb4hpc

4.16.5 (D)

4.16.5 (D)

4.16.4 (D)

4.16.4 (D)

4.16.3 (D)

4.16.3 (D)

4.16.2 (D)

intel

lmod_scripts

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

nvhpc

24.3 (D)

nvidia

25.5 (D)

25.5 (D)

24.3 (D)

24.3 (D)

24.3 (D)

24.3 (D)

24.3 (D)

perftools

25.09.0 (D)

25.09.0 (D)

25.03.0 (D)

25.03.0 (D)

24.11.0 (D)

24.11.0 (D)

24.07.0 (D)

rocm

6.3.0

6.3.0

6.2.1

6.2.1

6.1.0 (D)

saniti-zers4hpc

1.1.6 (D)

1.1.6 (D)

1.1.5 (D)

1.1.5 (D)

1.1.4 (D)

1.1.4 (D)

1.1.3 (D)

total-viewsup

2024.4.0

2024.4.0

2024.1.21

2024.1.21

2024.1.21

val-grind4hpc

2.13.6 (D)

2.13.6 (D)

2.13.5 (D)

2.13.5 (D)

2.13.4 (D)

2.13.4 (D)

2.13.3 (D)

SLES X86 support matrix

SLES on X86 systems is supported with CPE on HPE Cray Supercomputing EX systems with either CSM or HPCM. Below are product components, modules, third-party software versions supported with previous CPE releases with these configurations.

(D) represents the default version installed at installation.

* HPCM only

Release

CPE 25.09

CPE 25.09

CPE 25.03

CPE 25.03

24.11

24.11

24.07

Product/Version

sles15sp7 *

sles15sp6

sles15sp6

sles15sp5

sles15sp6

sles15sp5

sles15sp5

COS

25.9

25.9

25.1

24.7

24.7

COS Base

N/A

N/A

3.3.0

3.1.0

3.2.0

3.1.0

3.1.0

CSM

Not supported

1.7

1.6.1

1.6.1

1.6

1.6

1.5

HPCM

1.14

1.14

1.13

1.13

1.12

1.12

1.11

USS

1.4.0

1.4.0

1.3.0

1.1.0

1.2.0

1.1.0

1.1.0

amd

6.4.1 (D)

6.4.1 (D)

6.2.1 (D)

6.2.1 (D)

6.2.1 (D)

6.2.1 (D)

aocc

5.0.0 (D)

5.0.0 (D)

5.0.0 (D)

5.0.0 (D)

4.2.0 (D)

4.2.0 (D)

4.2.0 (D)

atp

3.15.7 (D)

3.15.7 (D)

3.15.6 (D)

3.15.6 (D)

3.15.5 (D)

3.15.5 (D)

3.15.4 (D)

cce

20.0.0

20.0.0

19.0.0 (D)

19.0.0 (D)

18.0.1 (D)

18.0.1 (D)

18.0.0 (D)

cpe-gcc-mpfr

3.1.4 (D)

3.1.4 (D)

3.1.4 (D)

3.1.4 (D)

3.1.4 (D)

cpe-gcc-native

14.2 (D)

14.2 (D)

cpe-gcc-native

14 (D)

14

13.2

13.2

13.2 (D)

13.2 (D)

13.2 (D)

cpe-gcc-native

13

12.3

12.3

12.3

12.3

12.3

cpe-gcc-native

12

12.3

cpe-prgenv-amd

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-aocc

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-cray

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-cray-amd

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-gnu

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-gnu-amd

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-intel

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.5.0 (D)

cpe-prgenv-nvidia

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

8.6.0 (D)

4.4.0 (D)

4.4.0 (D)

8.5.0 (D)

cray-R

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

4.4.0 (D)

cray-ccdb

5.0.7 (D)

5.0.7 (D)

5.0.6 (D)

5.0.6 (D)

5.0.5 (D)

5.0.5 (D)

5.0.4 (D)

cray-cdst-support

2.14.6 (D)

2.14.6 (D)

2.14.5 (D)

2.14.5 (D)

2.14.3 (D)

cray-cti

2.20.0 (D)

2.20.0 (D)

2.19.1 (D)

2.19.1 (D)

2.19.0 (D)

2.19.0 (D)

2.18.4 (D)

cray-dsmml

0.3.1 (D)

0.3.1 (D)

0.3.1 (D)

0.3.1 (D)

0.3.0 (D)

0.3.0 (D)

0.3.0 (D)

cray-dwarf

2.0.0 (D)

2.0.0 (D)

0.11.1 (D)

0.11.1 (D)

0.11.0 (D)

0.11.0 (D)

0.9.2 (D)

cray-dyninst

12.3.6 (D)

12.3.6 (D)

12.3.5 (D)

12.3.5 (D)

12.3.4 (D)

12.3.4 (D)

12.3.2 (D)

cray-fftw

3.3.10.11 (D)

3.3.10.11 (D)

3.3.10.10 (D)

3.3.10.10 (D)

3.3.10.9 (D)

3.3.10.9 (D)

3.3.10.8 (D)

cray-hdf5

1.14.3.7 (D)

1.14.3.7 (D)

1.14.3.5 (D)

1.14.3.5 (D)

1.14.3.3 (D)

1.14.3.3 (D)

1.14.3.1 (D)

cray-libsci

25.09.0 (D)

25.09.0 (D)

25.03.0 (D)

25.03.0 (D)

24.11.0 (D)

24.11.0 (D)

24.07.0 (D)

cray-libsci-acc

25.09.0 (D)

25.09.0 (D)

25.03.0 (D)

25.03.0 (D)

24.11.0 (D)

24.11.0 (D)

24.07.0 (D)

cray-lmod

8.7.60 (D)

8.7.60 (D)

8.7.55 (D)

8.7.55 (D)

8.7.37 (D)

8.7.37 (D)

8.7.37 (D)

cray-modules

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

3.2.11.7 (D)

cray-mpich

8.1.33

8.1.33

8.1.32 (D)

8.1.32 (D)

8.1.31 (D)

8.1.31 (D)

8.1.30 (D)

cray-mpich

9.0.1

9.0.1

9.0.0

9.0.0

cray-mpixlate

1.0.7 (D)

1.0.7 (D)

1.0.6 (D)

1.0.6 (D)

1.0.5 (D)

cray-mrnet

5.1.6 (D)

5.1.6 (D)

5.1.5 (D)

5.1.5 (D)

5.1.4 (D)

5.1.4 (D)

5.1.3 (D)

cray-netcdf

4.9.2.1 (D)

4.9.2.1 (D)

4.9.0.17 (D)

4.9.0.17 (D)

4.9.0.15 (D)

4.9.0.15 (D)

4.9.0.13 (D)

cray-open-shmemx

11.7.5 (D)

11.7.5 (D)

11.7.4 (D)

11.7.4 (D)

11.7.3 (D)

11.7.3 (D)

11.7.2 (D)

cray-pals

1.3.2

cray-papi

7.2.0.2 (D)

7.2.0.2 (D)

7.2.0.1 (D)

7.2.0.1 (D)

7.1.0.4 (D)

7.1.0.4 (D)

7.1.0.2 (D)

cray-parallel-netcdf

1.12.3.19 (D)

1.12.3.19 (D)

1.12.3.17 (D)

1.12.3.17 (D)

1.12.3.15 (D)

1.12.3.15 (D)

1.12.3.13 (D)

cray-pe-set-default

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

3.3 (D)

cray-pmi

6.1.16 (D)

6.1.16 (D)

6.1.15 (D)

6.1.15 (D)

6.1.15 (D)

6.1.15 (D)

6.1.15 (D)

cray-pmi-devel

6.1.16

6.1.16

6.1.15

6.1.15

6.1.15

6.1.15

6.1.15

cray-pmi-doc

6.1.16

6.1.16

6.1.15

6.1.15

6.1.15

6.1.15

6.1.15

cray-python

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

3.11.7 (D)

cray-stat

4.12.6 (D)

4.12.6 (D)

4.12.5 (D)

4.12.5 (D)

4.12.4 (D)

4.12.4 (D)

4.12.3 (D)

cray-ucx

2.12.0 (D)

2.12.0 (D)

2.12.0 (D)

2.12.0 (D)

2.12.0 (D)

cray-zmqnet

1.3.2 (D)

1.3.2 (D)

1.3.1 (D)

1.3.1 (D)

1.0.0 (D)

1.0.0 (D)

craype

2.7.35 (D)

2.7.35 (D)

2.7.34 (D)

2.7.34 (D)

2.7.33 (D)

2.7.33 (D)

2.7.32 (D)

craype-dl-plugin-ftr

22.06.1.2 (D)

22.06.1.2 (D)

22.06.1.2 (D)

22.06.1.2 (D)

22.06.1.2 (D)

craype-dl-plugin-py3

21.04.1

21.04.1

21.04.1

21.04.1

21.04.1

craype-dl-plugin-py3

22.06.1.2

22.06.1.2

22.06.1.2

22.06.1.2

22.06.1.2

craype-dl-plugin-py3

22.08.1

22.08.1

22.08.1

22.08.1

22.08.1

craype-dl-plugin-py3

22.09.1

22.09.1

22.09.1

22.09.1

22.09.1

craype-dl-plugin-py3

22.12.1

22.12.1

22.12.1

22.12.1

22.12.1

craype-dl-plugin-py3

23.09.1

23.09.1

23.09.1

23.09.1

23.09.1

craype-dl-plugin-py3

24.03.1 (D)

24.03.1 (D)

24.03.1 (D)

24.03.1 (D)

24.03.1 (D)

craype-targets-ex

1.16.0 (D)

1.16.0 (D)

1.15.1 (D)

1.15.1 (D)

1.15.0 (D)

1.15.0 (D)

1.13.2 (D)

craypkg-gen

1.3.36 (D)

1.3.36 (D)

1.3.35 (D)

1.3.35 (D)

1.3.34 (D)

1.3.34 (D)

1.3.33 (D)

forgesup

24.1.1

24.1.1

23.1.2

23.1.2

23.1.2

gdb4hpc

4.16.5 (D)

4.16.5 (D)

4.16.4 (D)

4.16.4 (D)

4.16.3 (D)

4.16.3 (D)

4.16.2 (D)

intel

2025.1 (D)

2025.1 (D)

2025.0 (D)

2025.0 (D)

2024.2 (D)

2024.2 (D)

2024.0 (D)

lmod_scripts

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

3.2.1 (D)

nvhpc

24.3 (D)

nvidia

25.5 (D)

25.5 (D)

24.3 (D)

24.3 (D)

24.3 (D)

24.3 (D)

24.3 (D)

perftools

25.09.0 (D)

25.09.0 (D)

25.03.0 (D)

25.03.0 (D)

24.11.0 (D)

24.11.0 (D)

24.07.0 (D)

rocm

6.4.1 (D)

6.4.1 (D)

6.3.0

6.3.0

6.2.1

6.2.1

6.1.0 (D)

saniti-zers4hpc

1.1.6 (D)

1.1.6 (D)

1.1.5 (D)

1.1.5 (D)

1.1.4 (D)

1.1.4 (D)

1.1.3 (D)

total-viewsup

2024.4.0

2024.4.0

2024.1.21

2024.1.21

2024.1.21

val-grind4hpc

2.13.6 (D)

2.13.6 (D)

2.13.5 (D)

2.13.5 (D)

2.13.4 (D)

2.13.4 (D)

2.13.3 (D)

Documentation and support

Documentation is available as a resource for using and managing CPE. This chapter provides details for obtaining CPE support and accessing available resources.

CPE installation and getting started guides

HPE CPE documentation comprises user and installation guides:

Title

Document Part Number

HPE Cray Supercomputing Programming Environment Installation Guide: CSM on HPE Cray Supercomputing EX Systems

S-8003

HPE Cray Supercomputing Programming Environment Installation Guide: HPCM on HPE Cray Supercomputing EX and HPE Cray Supercomputing Systems

S-8022

HPE Cray Supercomputing Programming Environment Installation Guide: HPE Cray XD2000 Systems

S-8012

HPE Cray Supercomputing Programming Environment Getting Started User Guide: HPE Cray Supercomputing EX Systems

S-9934

HPE Cray Supercomputing Programming Environment Getting Started Administrator Guide: HPE Cray Supercomputing EX Systems

S-9935

Other documentation resources

HPE provides CPE documentation and support through various online sources:

Published: April 2026