intro_pmi

intro_pmi - Introduces the Process Management Interface (PMI)

IMPLEMENTATION

Cray Linux Environment (CLE)

DESCRIPTION

The Process Management Interface (PMI) supports launching and managing the processes that make up the execution of a parallel program. PMI provides a scalable means for processes to communicate with each other and also allows different process managers to interact with each other in a standardized way.

PMI is normally transparent to the user. Direct PMI calls are most useful when experimenting with or simulating non-Cray network communications devices, or when developing an application launcher.

ROUTINES

The following PMI functions are typically used in application codes. Each function is documented in its own man page.

General Functions

PMI_Get_version_info

PMI2_Init

PMI2_Initialized

PMI2_Finalize

PMI2_Abort

PMI_Get_size

PMI_Get_rank

PMI_Get_universe_size

PMI_Get_appnum

Collective Operations

PMI_Allgather

PMI_Allgatherv

PMI_Barrier

PMI_Bcast

MPMD Operations

PMI_Get_rank_in_app

PMI_Get_base_rank_in_app

PMI_Get_app_size

PMI_Get_num_apps

PMI_Get_numpes_in_app_on_smp

PMI_Get_pes_in_app_on_smp

Topology Operations

PMI_Get_numpes_on_smp

PMI_Get_pes_on_smp

PMI_Get_nid

PMI_Get_nidlist_ptr

ENVIRONMENT VARIABLES

The following environment variables affect PMI behavior.

PMI_IP_ADDR_FAMILY

Cray PMI supports consistent usage of IPv4 and/or IPv6 addresses, within the system. Mixed usage of IP address types (where some interfaces on some nodes use IPv4 and others use IPv6) and IPv4-mapped-IPv6-addresses are not supported.

The recommended configuration uses only one IP address type (either IPv4 or IPv6), consistently, on all interfaces on all nodes. When so configured, Cray PMI will automatically detect the IP address type being used.

Configuration of both IPv4 and IPv6 addresses on all network interfaces on nodes within the system is also supported. With such configurations, by default, Cray PMI will use the first valid address type that is detected.

The address type to be used may be explicitly specified by setting PMI_IP_ADDR_FAMILY to either one of IPv4 or IPv6 (without the quotes). If this variable is set, addresses of that type must be configured on all interfaces within the system.

Default: unset

PMI_MMAP_SYNC_WAIT_TIME

This environment variable sets the length of seconds PMI waits before returning a timeout error during process launch. By default, PMI expects all requested processes to be launched on any given node within 180 seconds of each other. If the time span is longer than the value specified by PMI_MMAP_SYNC_WAIT_TIME, PMI assumes an error has occurred, and a timeout error is returned. Note that each node has its own timer and does not depend on total ranks in the job.

Default: 180

PMI_SIGNAL_STARTUP_COMPLETION

If set to a non-null value, this causes PMI to notify the user when the PMI initialization routine has completed on all ranks, by printing the message, READY TO ATTACH. This is useful for letting users know when they can successfully attached to the running process with a debugger, such as lgdb.

Default: unset

PMI_VERSION_DISPLAY

If set, causes PMI to display the CRAY PMI version number and build date information.

Default: not enabled

PMI_SHARED_SECRET

This is an integer value set by PALS for PMI and is secret for the launch. It enables PMI to authenticate internal networking connections. If this value is not set then PMI will attempt to authenticate using Munge or some other mechanism.

Default: Set by PALS for PMI

PMI_SPAWN_SRUN_ARGS

When the PMI Spawn API is used on a Slurm system, this environment variable controls the special arguments passed to Slurm’s srun to help it allocate from the available job slots and place the new ranks.

Default: –mpi=cray_shasta –exclusive