Lmod Custom Dynamic Hierarchy

Lmod enables a user to dynamically modify their user environment through Lua modules. The CPE implementation of Lmod capitalizes on its hierarchical structure, including the Lmod module auto-swapping functionality. This capability means that module dependencies determine the branches of the tree-like hierarchy. Lmod allows static and dynamic hierarchical module paths. Lmod provides full support for static paths, which build the hierarchy based on the current set of modules loaded. Alongside static paths, CPE implements dynamic paths for a subset of the Lmod hierarchy (compilers, networks, CPUs, and MPIs). Dynamic paths give an advanced level of flexibility for detecting multiple dependency paths and allow custom paths to join existing Lmod hierarchy in CPE without modifying customer module-files.

Static Lmod Hierarchy

Modules dependent on one or more modules being loaded are not visible to a user until their prerequisite modules are loaded. When the prerequisite modules are loaded, it adds the static paths of the dependent modules to the MODULEPATH environment variable, thereby exposing the dependent modules to the user. For more detailed information on the Lmod static module hierarchy, consult the Lmod Documentation listed in the “Other Module Resources” section above.

Dynamic Lmod Hierarchy

The CPE Lmod custom dynamic hierarchy abbreviates the overall Lmod hierarchy tree by relying on compatibility and not directly on a prerequisite version. Therefore, dependent modules do not need to exist in a new branch every time their prerequisite modules change versions. Instead, dynamic paths use a compatibility version that increases when a new prerequisite module version breaks compatibility in some way. The number following the path alias of the module (for example, 1.0 in x86-rome/1.0 and ofi/1.0) identifies the compatible version.

Module Path Aliases and Current Compatibility Versions

Compatible versions listed in the following tables include the minimum supported versions.

Compiler

RHEL Module Alias/Compatible Version

SLES Module Alias/Compatible Version

amd

amd/4.0

amd/4.0

cce

crayclang/16.0

crayclang/17.0

gcc

gnu/10.0

gnu/12.0

aocc

aocc/4.1

aocc/4.1

intel

intel/2023.2

intel/2023.2

nvidia

nvidia/20

nvidia/20

Network

Module Alias/Compatible Version

craype-network-none

none/1.0

craype-network-ofi

ofi/1.0

craype-network-ucx

ucx/1.0

CPU

Module Alias/Compatible Version

craype-x86-milan

x86-milan/1.0

craype-x86-rome

x86-rome/1.0

craype-x86-trento

x86-trento/1.0

MPI

Module Alias/Compatible Version

cray-mpich

cray-mpich/8.0

cray-mpich-abi

cray-mpich/8.0

cray-mpich-abi-pre-intel-5.0

cray-mpich/8.0

cray-mpich-ucx

cray-mpich/8.0

cray-mpich-ucx-abi

cray-mpich/8.0

cray-mpich-ucx-abi-pre-intel-5.0

cray-mpich/8.0

Custom Dynamic Hierarchy

The CPE custom dynamic hierarchy extension allows custom module paths to join the existing CPE Lmod hierarchy implementation without modifying customer modulefiles. The custom dynamic module types that CPE supports include:

  • Compiler

  • Network

  • CPU

  • MPI

  • Compiler/Network

  • Compiler/CPU

  • Compiler/Network/CPU/MPI

As each custom dynamic module type loads, a handshake occurs using special pre-defined environment variables. When all hierarchical prerequisites are met, the paths of the dependent modulefiles are added to the MODULEPATH environment variable, thereby exposing the dependent modules to the user.

For Lmod to assist a user optimally, load the compiler, network, CPU, and MPI module. Lmod cannot detect modules hidden in dynamic paths without one of each type of module being loaded.

Create a custom dynamic hierarchy

PREREQUISITES

Set Lmod as the default module handling system before initiating this procedure.

OBJECTIVE

For the CPE custom dynamic hierarchy to detect the desired Lmod module path, one or more custom dynamic environment variables must be created according to the requirements defined within this procedure.

PROCEDURE

To create a custom dynamic environment variable:

  1. Begin the environment variable name with LMOD_CUSTOM_

  2. Append the descriptor of the module type that the environment variable will represent. The module types and descriptors are:

Module Type

Descriptor

Compiler

COMPILER_

Network

NETWORK_

CPU

CPU_

MPI

MPI_

Compiler/Network

COMNET_

Compiler/CPU

COMCPU_

Compiler/Network/CPU/MPI

CNCM_

Example: The custom dynamic environment variable for the combined compiler and CPU module begins with LMOD_CUSTOM_COMCPU_.

  1. Following the descriptor, append all prerequisite module aliases along with their respective compatible versions. See Module Path Aliases and Current Compatibility Versions for more information. The format of the module path alias/compatible version string for each module type is shown below.

Module Type: Module Path Alias/Compatible Version String

Compiler: <compiler_name>/<compatible_version>

Network: <network_name>/<compatible_version>

CPU: <cpu_name>/<compatible_version>

MPI: <compiler_name>/<compatible_version>/<network_name>/<compatible_version>/ <mpi_name>/<compatible_version>

Compiler/Network: <compiler_name>/<compatible_version/<network_name>/<compatible_version>

Compiler/CPU: <compiler_name>/<compatible_version>/<cpu_name>/<compatible_version>

Compiler/Network/CPU/MPI: <compiler_name>/<compatible_version>/<network_name>/<compatible_version>/ <cpu_name>/<compatible_version>/<mpi_name>/<compatible_version>

To create an acceptably formatted environment variable name, replace all slashes and dots in the module alias/compatible version string with underscores. Also, all letters must be in uppercase format.

Example Module Path Alias/Compatible Version Strings:

  • Compiler = cce

    The path alias/compatible version string (values found in Module Path Aliases and Current Compatibility Versions) is crayclang/10.0; therefore, the text added to the environment variable name is:

    CRAYCLANG_10_0

  • Network = craype-network-ofi

    The path alias/compatible version string is ofi/1.0; therefore, the environment variable text is:

    OFI_1_0

  • CPU = craype-x86-rome

    The path alias/compatible version string is x86-rome/1.0; therefore, the environment variable text is:

    X86_ROME_1_0

  • MPI = cray-mpich

    cray-mpich has two prerequisite module types (compiler and network). Therefore, the environment variable must include the alias/compatible version for the desired compiler, network, and MPI. For a cray-mpich module dependent on cce and craype-network-ofi, the path alias/compatible version string is crayclang/10.0/ofi/1.0/cray_mpich/8.0; therefore, the environment variable text is:

    CRAYCLANG_10_0_OFI_1_0_CRAY_MPICH_8_0

  • Compiler/Network = ccewithcraype-network-ofi

    The path alias/compatible version string is crayclang/10.0/ofi/1.0; therefore, the environment variable text is:

    CRAYCLANG_10_0_OFI_1_0

  • Compiler/CPU = cce with craype-x86-rome

    The path alias/compatible version string is crayclang/10.0/x86-rome/1.0; therefore, the environment variable text is:

    CRAYCLANG_10_0_X86_ROME_1_0

  • Compiler/Network/CPU/MPI = cce,craype-network-ofi,craype-x86-rome,andcray-mpich

    The path alias/compatible version string is crayclang/10.0/ofi/1.0/x86-rome/1.0/cray-mpich/8.0; therefore, the envi- ronment variable text is:

    CRAYCLANG_10_0_OFI_1_0_X86_ROME_1_0_CRAY_MPICH_8_0

  1. Append _PREFIX following the final module/compatibility text instance:

Example: Network = craype-network-ofi

The custom dynamic environment variable is LMOD_CUSTOM_NETWORK_OFI_1_0_PREFIX.

Creation of the custom dynamic environment variable is now complete.

  1. Add the custom dynamic environment variable to the user environment by exporting it with its value set to the Lmod module path:

# export LMOD_CUSTOM_NETWORK_OFI_1_0_PREFIX=<lmod_module_path>

Example: Network = craype-network-ofi

All modulefiles in <lmod_module_path> are shown to users whenever craype-network-ofi is loaded.