About the HPE CPE installation guide: Apollo 2000 Gen10 Plus (x86) systems

The HPE Cray Programming Environment (CPE) for Apollo 2000 Gen10 Plus (x86) systems is distributed as a collection of RPM packages. This publication documents the installation of RPMs and related files on these systems.

This publication is intended for system administrators receiving their first release of this product or upgrading from a previous release. The information in this guide assumes that the administrator has a good understanding of Linux system administration.

Release information

This publication supports the installation of the HPE CPE 24.07 release on Apollo 2000 Gen10 Plus (x86) systems running RHEL 8.10 with HPCM 1.11. The CPE 24.07 release supports Mellanox OFED (MOFED) networking stack 5.8.

Variable substitutions

Variable substitutions to be used throughout this guide include:

  • <CPE_RELEASE> = 24.07

  • <AOCC_VERSION> = 4.2.0

  • <INTEL_VERSION> = 2024.0.1

  • <MOFED_VERSION> = 5.8

  • <IMAGE_ROOT> = /opt/clmgr/image/images/<IMAGE_NAME>, where <IMAGE_NAME> is chosen by the site

Record of revision

New in the CPE 24.07 publication

New in the CPE 24.03 publication

New in the CPE 23.12 publication

New in the CPE 23.09 publication

New in the CPE 23.05 publication

New in the CPE 23.02 (Rev. A) publication

New in the CPE 23.02 publication

  • Updated release information and incorporated editorial updates.

Publication Title

Date

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (24.07) S-8012

August 2024

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (24.03) S-8012

May 2024

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (23.12) S-8012

December 2023

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (23.09) S-8012

September 2023

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (23.05) S-8012

June 2023

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (23.02-Rev. A) S-8012

March 2023

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (23.02) S-8012

February 2023

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.12) S-8012

December 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.11) S-8012

November 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.10) S-8012

October 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.09) S-8012

September 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.08) S-8012

August 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.06) S-8012

June 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.05) S-8012

May 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.04) S-8012

April 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.03) S-8012

March 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (22.02) S-8012

February 2022

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (21.02 - 21.12) S-8012

Feb - Dec 2021

HPE Cray Programming Environment for the Apollo 2000 Gen10 Plus (x86) System Installation Guide (20.08, 20.09, 20.11) S-8012

Aug, Sep, Nov 2020

Command Prompt Conventions

Host name and account in command prompts: The host name in a command prompt indicates where the command must be run. The account that must run the command is also indicated in the prompt.

  • The root or super-user account always has the # character at the end of the prompt.

  • Any non-root account is indicated with account@hostname>. A user account that is neither root nor crayadm is seen as user.

Command Prompt

Definition

user@login>

Run the command on any login node as any non-root user.

hostname#

Run the command on the specified system as root.

user@hostname>

Run the command on the specified system as any non-root user.

Downloading HPE Cray Supercomputing EX software

To download HPE Cray Supercomputing EX software, refer to the HPE Support Center or download it directly from My HPE Software Center. The HPE Support Center contains a wealth of documentation, training videos, knowledge articles, and alerts for HPE Cray Supercomputing EX systems. It provides the most detailed information about a release as well as direct links to product firmware, software, and patches available through My HPE Software Center.

Downloading the software through the HPE Support Center

HPE recommends downloading software through the HPE Support Center because of the many other resources available on the website.

  1. Visit the HPE Cray Supercomputing EX product page on the HPE Support Center.

  2. Search for specific product info, such as the full software name or recipe name and version.

    For example, search for “Slingshot 2.1” or “Cray System Software with CSM 24.3.0.”

  3. Find the desired software in the search results and select it to review details.

  4. Select Obtain Software and select Sign in Now when prompted.

    If a customer’s Entitlement Order Number (EON) is tied to specific hardware rather than software, the software is available without providing account credentials. Access the software instead by selecting Download Software and skip the next step in this procedure.

  5. Enter account credentials when prompted and accept the HPE License Terms.

    To download software, customers must ensure their Entitlement Order Number (EON) is active under My Contracts & Warranties on My HPE Software Center. If customers have trouble with the EON or are not entitled to a product, they must contact their HPE contract administrator or sales representative for assistance.

  6. Choose the needed software and documentation files to download and select curl Copy to access the files.

    Just like the software files, the documentation files change with each release. In addition to the official documentation, valuable information for a release is often available in files that include the phrase README in their name. Be sure to select and review these files in detail.

    HPE recommends the curl Copy option, which downloads a single text file with curl commands to use on the desired system. You must run the curl commands within 24 hours of downloading them or download new commands if more than 24 hours have passed.

    To validate the security of the downloads, you can later compare the files on the desired system against the checksums provided by HPE underneath each selected download.

  7. Save the text file to a central location.

  8. On the system where the software will be downloaded, run a shell script to execute the text file that includes the curl commands.

    For example:

    ncn-m001# bash -x <TEXT_FILE_PATH>
    

    The -x option in this example tracks the download progress of each curl command in the text file.

Downloading the software directly from the My HPE Software Center

Users already familiar with a release can save time by downloading software directly from My HPE Software Center.

  1. Visit My HPE Software Center and select Sign in.

  2. Enter account credentials when prompted and select Software in the left navigation bar.

  3. Search for specific product info, such as the full software name or recipe name and version.

    For example, search for “Slingshot 2.1” or “Cray System Software with CSM 24.3.0.”

  4. Find the desired software in the search results and review details by selecting Product Details under the Action column.

    Image of Product Detailsoption

  5. Select Go To Downloads Page and accept the HPE License Terms.

    To download software, customers must ensure their Entitlement Order Number (EON) is active under My Contracts & Warranties. If customers have trouble with the EON or are not entitled to a product, they must contact their HPE contract administrator or sales representative for assistance.

  6. Choose the needed software and documentation files to download and select curl Copy to access the files.

    Just like the software files, the documentation files change with each release. In addition to the official documentation, valuable information for a release is often available in files that include the phrase README in their name. Be sure to select and review these files in detail.

    HPE recommends the curl Copy option, which downloads a single text file with curl commands to use on the desired system. You must run the curl commands within 24 hours of downloading them or download new commands if more than 24 hours have passed.

    To validate the security of the downloads, you can later compare the files on the desired system against the checksums provided by HPE underneath each selected download

  7. Save the text file to a central location.

  8. On the system where the software will be downloaded, run a shell script to execute the text file that includes the curl commands.

    For example:

    ncn-m001# bash -x <TEXT_FILE_PATH>
    

    The -x option in this example tracks the download progress of each curl command in the text file.

Install HPE CPE on Apollo 2000 Gen10 Plus Systems with HPCM

PREREQUISITES

  • A cluster image for use with CPE must already be present on the system. You can optionally clone the current cluster image to a new one to be used by CPE or save a revision of the current cluster image before continuing with this procedure. Consult the HPE Performance Cluster Manager Administrator’s Guide for more information on HPCM management commands.

  • The glibc-devel package must be installed.

  • Mellanox OFED must be updated to <MOFED_VERSION> version. See Release Information for additional details.

  • If you are using Lua modulefiles, the RHEL CodeReady Builder repository must be enabled before installing cray-lmod.

OBJECTIVE

Use this procedure to install the HPE Cray Programming Environment into an image on the Apollo 2000 Gen10 Plus (x86) system using HPCM.

PROCEDURE

  1. Add the CPE repository from the ISO media:

    mgmt1# cm repo add <PATH_TO_ISO>
    
  2. Confirm the CPE repository is available, and note the name of the repository:

    mgmt1# cm repo show | grep Cray
    Cray-Programming-Environment-for-apollo2000-<CPE_RELEASE>-rhel8-x86_64 : \
    /opt/clmgr/repos/cm/Cray-Programming-Environment-for-apollo2000-<CPE_RELEASE>-rhel8-x86_64
    
  3. Select the CPE repository using the name shown in the previous command:

    mgmt1# cm repo select \
    Cray-Programming-Environment-for-apollo2000-<CPE_RELEASE>-rhel8-x86_64
    
  4. Install CPE base content into the desired image.

    mgmt1# cm image dnf -i <IMAGE_NAME> install cpe-apollo2000-base-content-<CPE_RELEASE>
    
  5. Install the appropriate package for the system CPU target (rome or milan):

    mgmt1# cm image dnf -i <IMAGE_NAME> install cray-pe-profile-ofi-<CPU_TARGET>
    
  6. (Optional) Set CrayPE as default on the system:

    mgmt1# chroot <IMAGE_ROOT> sed -i '/^#.*enabled_by_default/s/^#//' \
    /etc/cray-pe.d/cray-pe-configuration.*sh
    
  7. (Optional) Install GCC version 12 support if your operating system is RHEL 8.7:

    mgmt1# cm image dnf -i <IMAGE_NAME> install gcc-native-12.1
    
  8. (Optional) Install the CPE AOCC support content.

    Note: CPE does not contain the AOCC compiler itself; the compiler must be installed separately.

    mgmt1# cm image dnf -i <IMAGE_NAME> install cpe-apollo2000-aocc-content-<CPE_RELEASE>
    
  9. (Optional) Install the CPE Intel oneAPI support content and generate the modulefiles.

    Note: CPE does not contain the Intel oneAPI compiler itself; the compiler must be installed separately.

    mgmt1# cm image dnf -i <IMAGE_NAME> install cpe-apollo2000-intel-content-<CPE_RELEASE>
    mgmt1# chroot <IMAGE_ROOT> 
    mgmt1(chroot)# . /opt/cray/pe/modules/default/init/bash
    mgmt1(chroot)# module load craypkg-gen
    mgmt1(chroot)# craypkg-gen -m /opt/intel/oneapi/compiler/<INTEL_VERSION>
    mgmt1(chroot)# exit
    
  10. (Optional) If you are using Lua modulefiles, install cray-lmod:

    mgmt1# cm image dnf -i <IMAGE_NAME> install cray-lmod lua-posix lua-filesystem
    mgmt1# chroot <IMAGE_ROOT> sed -i 's/"environment modules"/"lmod"/' \
    /etc/cray-pe.d/cray-pe-configuration.*sh
    
  11. Set the release default:

    mgmt1# chroot <IMAGE_ROOT> /opt/cray/pe/cpe/<CPE_RELEASE>/set_default_cpe_<CPE_RELEASE>
    
  12. Stage the image on the desired nodes:

    mgmt1# cm node provision -i <IMAGE_NAME> --stage --nodes n0*
    
  13. Reboot the nodes to use the image:

    mgmt1# cm power reset -t node n0*
    

Install HPE CPE on Apollo 2000 Gen10 Plus Systems without HPCM

PREREQUISITES

  • Python 3.6.0 must be installed. The cpe-installer requires Python 3.6.0.

  • A cluster image for use with CPE must be present on the system. You can optionally clone the current cluster image to a new one to be used by CPE or save a revision of the current cluster image before continuing with this procedure.

  • The glibc-devel package must be installed.

  • Mellanox OFED must be updated to <MOFED_VERSION> version. See Release Information for additional details.

  • If you are using Lua modulefiles, the RHEL CodeReady Builder repository must be enabled before installing cray-lmod.

OBJECTIVE

This procedure provides instructions for installing the HPE Cray Programming Environment into an image root on the Apollo 2000 Gen10 Plus (x86) system.

PROCEDURE

  1. Create the directory to mount the ISO file:

    mgmt1# mkdir CPE-apollo2000-<CPE_RELEASE>
    
  2. Mount the ISO file:

    mgmt1# mount -r -o loop <PATH_TO_ISO> CPE-apollo2000-<CPE_RELEASE>
    
  3. Change directory to the ISO mount directory:

    mgmt1# cd CPE-apollo2000-<CPE_RELEASE>
    
  4. Run the installer with the --cpu-target (a supported CPU target architecture) and --image-root parameters, with a full path to the image root:

    mgmt1# ./cpe-installer.py --cpu-target {x86-rome, x86-milan} --image-root <IMAGE_ROOT>
    

    Example:

    Add the --include-env aocc parameter to install the CPE AOCC support content.

    Note: CPE does not contain the AOCC Compiler; the compiler must be installed separately.

    mgmt1# ./cpe-installer.py --cpu-target {x86-rome, x86-milan} \
    --image-root <IMAGE_ROOT> --include-env aocc
    

    Example:

    Add the --include-env intel parameter to install the CPE Intel oneAPI support content.

    Note: CPE does not contain the Intel oneAPI compiler; the compiler must be installed separately.

    mgmt1# ./cpe-installer.py --cpu-target {x86-rome, x86-milan} \
    --image-root <IMAGE_ROOT> --include-env intel
    
  5. (Optional) Install GCC version 12 support if your operating system is RHEL 8.7:

    mgmt1# dnf install --installroot <IMAGE_ROOT> gcc-native-12.1
    
  6. (Optional) Generate the modulefiles for Intel if you are installing the CPE Intel oneAPI support content:

    mgmt1# chroot <IMAGE_ROOT> 
    mgmt1(chroot)# . /opt/cray/pe/modules/default/init/bash
    mgmt1(chroot)# module load craypkg-gen
    mgmt1(chroot)# craypkg-gen -m /opt/intel/oneapi/compiler/<INTEL_VERSION>
    mgmt1(chroot)# exit
    
  7. (Optional) Set CrayPE as default on the system:

    mgmt1# chroot <IMAGE_ROOT> sed -i '/^#.*enabled_by_default/s/^#//' cray-pe-configuration.*sh
    
  8. (Optional) If you are using Lua modulefiles, get the CPE-apollo2000-<CPE_RELEASE> repo ID, and then install cray-lmod:

    mgmt1# dnf repolist
    mgmt1# dnf install --config /etc/yum.repos.d/<REPO_ID>.repo \
    --installroot <IMAGE_ROOT> cray-lmod lua-posix lua-filesystem
    
    mgmt1# chroot <IMAGE_ROOT> sed -i \
    's/"environment modules"/"lmod"/' /etc/cray-pe.d/cray-pe-configuration.*sh
    
  9. Set the release default:

    mgmt1# chroot <IMAGE_ROOT> /opt/cray/pe/cpe/<CPE_RELEASE>/set_default_cpe_<CPE_RELEASE>
    
  10. Unmount the ISO:

    mgmt1# cd .. && umount CPE-apollo2000-<CPE_RELEASE>
    
  11. Reboot the compute nodes using the image with CPE installed.

Create Modulefiles for Third-Party Products

PREREQUISITES

Third-party packages must be downloaded and installed.

OBJECTIVE

These procedures provide instructions for creating modulefiles for third-party products. The instructions use craypkg-gen to create a modulefile for a specific version of a supported third-party product. This allows a site to set a specific version as default.

The following tasks are necessary and can be embedded in a script where a third-party product is being installed.

PROCEDURE

  1. Load craypkg-gen module:

    source /opt/cray/pe/modules/default/init/bash
    module use /opt/cray/pe/modulefiles
    module load craypkg-gen
    
  2. Generate module and set default scripts for products:

    AMD Optimizing C/C++ Compiler: (requires craypkg-gen >= 1.3.16)

    craypkg-gen -m /opt/AMD/aocc-compiler-<AOCC_VERSION>/
    

    Intel OneAPI

    craypkg-gen -m /opt/intel/oneapi/compiler/<INTEL_VERSION>/
    
  3. Run a set default script:

    /opt/admin-pe/set_default_craypkg/set_default_<MODULE_NAME>_<MODULE_VERSION>
    

Lmod Custom Dynamic Hierarchy

Lmod enables a user to dynamically modify their user environment through Lua modules. CPE capitalizes on the Lmod hierarchical structure, including the Lmod module auto-swapping functionality. This functionality allows module dependencies to determine the branches of the tree-like hierarchy. Lmod allows static and dynamic hierarchical module paths. Lmod provides full support for static paths, which build the hierarchy based on the current set of modules loaded. Alongside static paths, CPE implements dynamic paths for a subset of the Lmod hierarchy (compilers, networks, CPUs, and MPIs). Dynamic paths give an advanced level of flexibility for detecting multiple dependency paths and allow custom paths to join the existing CPE Lmod hierarchy without modifying customer modulefiles.

Static Lmod Hierarchy

Modules dependent on one or more modules being loaded are not visible to a user until their prerequisite modules are loaded. When the prerequisite modules are loaded, it adds the static paths of the dependent modules to the MODULEPATH environment variable, thereby exposing the dependent modules to the user. For more detailed information on Lmod’s static module hierarchy, please consult User Guide for Lmod.

Dynamic Lmod Hierarchy

The custom dynamic Lmod hierarchy of CPE abbreviates the overall Lmod hierarchy tree by relying on compatibility and not directly on a prerequisite’s version. Therefore, dependent modules do not need to exist in a new branch every time their prerequisite modules change versions. Instead, dynamic paths use a compatibility version that increases when a new prerequisite module version breaks compatibility in some way. The number following the module path alias (for example, 1.0 in x86-rome/1.0 and /ofi/1.0) identifies the compatible version.

Module Path Aliases and Current Compatibility Versions

Compatible versions listed in the following tables include the minimum supported versions.

Compiler

RHEL Module Alias/Compatible Version

amd

amd/4.0

cce

crayclang/16.0

aocc

aocc/4.1

intel

intel/2023.2

gcc

gnu/10.0

nvidia

nvidia/20

Network

Module Alias/Compatible Version

craype-network-infiniband

infiniband/1.0

craype-network-ofi

ofi/1.0

craype-network-ucx

ucx/1.0

CPU

Module Alias/Compatible Version

craype-broadwell

broadwell/1.0

craype-x86-milan

x86-milan/1.0

craype-x86-rome

x86-rome/1.0

craype-x86-skylake

x86-skylake/1.0

craype-x86-cascadelake

x86-cascadelake/1.0

craype-x86-icelake

x86-icelake/1.0

MPI

Module Alias/Compatible Version

cray-mpich

cray-mpich/8.0

cray-mpich-abi

cray-mpich/8.0

cray-mpich-abi-pre-intel-5.0

cray-mpich/8.0

cray-mpich-ucx

cray-mpich/8.0

cray-mpich-ucx-abi

cray-mpich/8.0

cray-mpich-ucx-abi-pre-intel-5.0

cray-mpich/8.0

Custom Dynamic Hierarchy

The CPE custom dynamic hierarchy extension allows custom module paths to join the existing CPE Lmod hierarchy implementation without modifying customer modulefiles. The custom dynamic module types CPE supports include:

  • Compiler

  • Network

  • CPU

  • MPI

  • Compiler/Network

  • Compiler/CPU

  • Compiler/Network/CPU/MPI

As each custom dynamic module type loads, a handshake occurs using special pre-defined environment variables. When all hierarchical prerequisites are met, the paths of the dependent modulefiles are added to the MODULEPATH environment variable, thereby exposing the dependent modules to the user.

For Lmod to assist a user optimally, load a compiler, network, CPU, and MPI module. Lmod cannot detect modules hidden in dynamic paths without one of each type of module being loaded.

Create a custom dynamic hierarchy

PREREQUISITES

Set Lmod as the default module handling system before initiating this procedure.

OBJECTIVE

For the CPE custom dynamic hierarchy to detect the desired Lmod module path, one or more custom dynamic environment variables must be created according to the requirements defined within this procedure.

PROCEDURE

To create a custom dynamic environment variable:

  1. Begin the environment variable name with LMOD_CUSTOM_.

  2. Append the descriptor of the module type that the environment variable will represent. The module types and descriptors are:

    Module Type

    Descriptor

    Compiler

    COMPILER_

    Network

    NETWORK_

    CPU

    CPU_

    MPI

    MPI_

    Compiler/Network

    COMNET_

    Compiler/CPU

    COMCPU_

    Compiler/Network/CPU/MPI

    CNCM_

    Example: The custom dynamic environment variable for the combined compiler and CPU module begins with LMOD_CUSTOM_COMCPU_.

  3. Following the descriptor, append all prerequisite module aliases along with their respective compatible versions. See Module Path Aliases and Current Compatibility Versions for more information. The format of the module path alias/compatible version string for each module type is shown below. Note that due to publishing issues, long module alias/compatible version strings are split across two lines as indicated below.

    Module Type: Module Path Alias/Compatible Version String

    Compiler: <compiler_name>/<compatible_version>

    Network: <network_name>/<compatible_version>

    CPU: <cpu_name>/<compatible_version>

    MPI:

    <compiler_name>/<compatible_version>/<network_name>/<compatible_version>/

    <mpi_name>/<compatible_version>

    Compiler/Network: <compiler_name>/<compatible_version/<network_name>/<compatible_version>

    Compiler/CPU: <compiler_name>/<compatible_version>/<cpu_name>/<compatible_version>

    Compiler/Network/CPU/MPI:

    <compiler_name>/<compatible_version>/<network_name>/<compatible_version>/

    <cpu_name>/<compatible_version>/<mpi_name>/<compatible_version>

    To create an acceptably formatted environment variable name, replace all slashes and dots in the module alias/compatible version string with underscores. Also, all letters must be in uppercase format.

    Example Module Path Alias/Compatible Version Strings:

  • Compiler = cce

    The path alias/compatible version string (values found in Module Path Aliases and Current Compatibility Versions) is crayclang/10.0; therefore, the text added to the environment variable name is:

    CRAYCLANG_10_0

  • Network = craype-network-ofi

    The path alias/compatible version string is ofi/1.0; therefore, the environment variable text is:

    OFI_1_0

  • CPU = craype-x86-rome

    The path alias/compatible version string is x86-rome/1.0; therefore, the environment variable text is:

    X86_ROME_1_0

  • MPI = cray-mpich

    cray-mpich has two prerequisite module types (compiler and network). Therefore, the environment variable must include the alias/compatible version for the desired compiler, network, and MPI. For a cray-mpich module dependent on cce and craype-network-ofi, the path alias/compatible version string is crayclang/10.0/ofi/1.0/cray_mpich/8.0; therefore, the environment variable text is:

    CRAYCLANG_10_0_OFI_1_0_CRAY_MPICH_8_0.

  • Compiler/Network = cce with craype-network-ofi

    The path alias/compatible version string is crayclang/10.0/ofi/1.0; therefore, the environment variable text is:

    CRAYCLANG_10_0_OFI_1_0

  • Compiler/CPU = cce with craype-x86-rome

    The path alias/compatible version string is crayclang/10.0/x86-rome/1.0; therefore, the environment variable text is:

    CRAYCLANG_10_0_X86_ROME_1_0

  • Compiler/Network/CPU/MPI = cce, craype-network-ofi, craype-x86-rome, and cray-mpich

    The path alias/compatible version string is crayclang/10.0/ofi/1.0/x86-rome/1.0/cray-mpich/8.0; therefore, the environment variable text is:

    CRAYCLANG_10_0_OFI_1_0_X86_ROME_1_0_CRAY_MPICH_8_0

  1. Append _PREFIX following the final module/compatibility text instance:

    Example: Network = craype-network-ofi

    The custom dynamic environment variable is LMOD_CUSTOM_NETWORK_OFI_1_0_PREFIX.

    Creation of the custom dynamic environment variable is now complete.

  2. Add the custom dynamic environment variable to the user environment by exporting it with its value set to the Lmod module path:

    # export LMOD_CUSTOM_NETWORK_OFI_1_0_PREFIX=<lmod_module_path>
    

    Example: Network = craype-network-ofi

    All modulefiles in <lmod_module_path> are shown to users whenever craype-network-ofi is loaded.