gdb4hpc Command Reference

This document contains the help text for each gdb4hpc command in gdb4hpc 4.16.4. It is the same material you could read when using the help command within gdb4hpc itself.

attach

Summary: Attach to an application.
Usage: attach <app_handle> <app_ident> [–gpu]
Additional options: [–debug[=<directory>]] [–gdb=<gdbapp>] [–gdb-data-dir=<datadir>]
Example command: attach $a 9544773

Arguments: app_handle (debugger variable syntax), app_ident (wlm specific
identifier of the running application)

Description:
Attach to an application that was previously launched outside of the debugger.

The app_handle argument is a process set handle used to refer to this application in future commands. The number of ranks is taken from the application, so app_handle should be scalar variable such as $a. After attaching, $a{0} refers to the first rank exactly like the launch case.

For ALPS, the apid can be obtained by using the apstat utility.

For Slurm, app_ident should be provided as <job_id>.<step_id> on Cray systems. The job_id is available from squeue and the step_id can be found using sstat –format=JobId <job_id>.

For Cray clusters or Apollo / Everest HPCM systems running PALS, app_ident should either be the PBS job ID for the PBS allocation, or the PALS app ID. If a PBS job ID is supplied, the first PALS application inside the allocation will be the attach target. To use a PALS application ID, start the attach inside the target PBS job, or set CTI_PALS_EXEC_HOST to the PBS job host (this can be found by running qstat -f).

–gpu enables GPU debugging. This will enable the use of a GPU supported version of gdb and set several environment variables for debugging.

–debug=<logdir> writes gdb4hpc logging information to a directory. If logdir is not specified,the enviroment variable $CRAY_DBG_LOG_DIR is used. The directory needs to be a location writable from the compute nodes.

–gdb=<gdbapp> directly specifies the gdb to use, <gdbapp> is either the full path to a gdb executable, or on application in PATH, for example: –gdb=cuda-gdb will use the locally installed cuda-gdb instead of the version tested and packaged with gdb4hpc. Use is experimental. –gdb-data-dir=<datadir> explicit data directory for the –gdb option –vgdb=<vgdb_file> will tell GDB4hpc to read Valgrind VGDB attach commands from the provided file. This is usually generated by Valgrind4hpc.

assign

Summary: Assignment command.
Usage: assign <target> <expr>
Example command: assign j 20

Arguments: target (expression syntax), expr (expression syntax)

Description:
Assign the value of an expression defined by expr to target. The target must evaluate to an l-value that is either a debugger variable (such as $a) or an application variable. If the target is the name of a debugger variable that does not exist, a new debugger variable with that name is automatically created.

The target can include a list of application ranks such as $a{1..3,5,10}. The assignment will occur as long as the expr evaluates into a single scalar, or the number of ranks in the expr result match the number of ranks in the target.

For example, if the value 10 is to be assigned myVar in ranks 1,3,4,5,10 of app_handle $a, the following assign command would be issued:

assign $a{1,3..5,10}::myVar 10

backtrace

Summary: Print stack frame backtrace command.
Usage: backtrace [<num>]
Example command: backtrace
Shortcut: bt

Arguments: num (pos/neg num frames)

Description:
Display a backtrace of the stack frames for every application rank in the current process set. If num is supplied, the innermost num frames are displayed. If num is negative, the outermost num frames are displayed.

break

Summary: Set breakpoint command.
Usage: break <location>
Usage: break <location> thread <thread-list>
Example command: break main.c:20
Example command: break main.c:20 thread 2-4
Shortcut: b

Arguments: location (line number, function name, or address)

Description:
Set a breakpoint in every application rank in the current process set. The location of the breakpoint can be defined as a line number, a function name, a source file:line-no, or * and an address. If an address is specified, it must begin with an asterisk *. For example, break *0x0000000000400693 would break at address 0x400693. Adding thread thread-set creates a thread specific breakpoint which will apply only to the given thread(s).

save breakpoints

Summary: Save breakpoint state to a file.
Usage: save breakpoints <filename>

Save the current breakpoints, watchpoints, and catchpoints to a file on disk.
Use the source command to restore them.

The saved file contains plaintext gdb4hpc commands and can be edited with any
text editor.

compare

Summary: Compare command.
Usage: compare <expr1> = <expr2>
Example command: compare $a{0,3..5,10}::i == $b{0,3..5,10}::i

Arguments: expr1 (variable expression), expr2 (variable expression)

Description:
Compare (subtract) the value of expression expr1 with the value of expression expr2. The values must be of the same size and shape, or a translation function (such as a decomposition) must be used. For example, if the command compare $a::myVar == $b::myVar is issued and application $a has 32 ranks and application $b has 64 ranks, the comparison is only valid if myVar has the same value in every rank of both $a and $b. If myVar has unique values in different ranks, the size and shape of the results do not match and the comparison will fail.

The output of this command depends on the settings of display, output, and error variables. For more complex expressions, the application variable part of the expression can be enclosed with quotes.

For example, the sizeof operator can be used by enclosing the syntax with quotes:

compare $a{0}::”sizeof(i)” = $b{0}::”sizeof(i)”

The app_handle can be subsets of ranks using the process set notation. When a subset of ranks is targeted, individual ranks are separated with a comma and a range of ranks is denoted with “..”. For example, if we wish to compare ranks 1,3,4,5,10 in $a, we would provide an app_handle of $a{1,3..5,10}.

continue

Summary: Continue debugging application.
Usage: continue [-a]
Example command: continue
Shortcut: c

Arguments: optional -a for “all”.

Description:
Continue execution of every application rank in the current process set from its current location. In non-stop mode, continue only applies to the current thread, use continue -a to continue execution on all threads.

cuda

Summary: Cuda debugging functions.
Usage: cuda <subcommand> [arguments]
Usage: cuda help <subcommand>
Usage: cuda info <subcommand>
Usage: cuda set <subcommand> <value>
Usage: cuda show <subcommand>

Calls the cuda thread functions, which will be available when launching/attaching
with –gpu and running with an nvidia device.
Use “cuda help” for the list of subcommands.

defset

Summary: Define process set command.
Usage: defset <name> <process set expression>
Example command: defset $new_set $a{0,1}
Example command: defset $thread_set $a{*:2..7}

Arguments: name (new process set name), process set expression (process
set expression list)

Description:
Create a new process set of application ranks that can be focused upon. The values of the process set expression must be valid launched/attached application names and valid ranks if a particular rank is targeted. A defined process set can contain ranks from two or more applications.

The process set expression should consist of a comma-separated list of debugger variable arrays specifying which ranks of the process set should be included in the new process set. A “..” can be used to denote a range of ranks.

For example, if the app_handle $a{1024} was provided to launch an application, $a{0..32,64} would create a process set containing ranks zero thru thirty-two and rank sixty-four of application $a. If the app_handle $b{2048} was provided to launch an application, $a{32},$b{32} would create a process set containing rank 32 of both applications.

The process set expression can optionally contain a thread set after a colon, for example defset $thread_set $a{0:2} will focus all actions onto thread 2 within rank 0. The thread expression is a comma separated list of thread ids, ranges of thread ids as <min-index>..<max-index>, and “<pattern>” for regular expressions that match on the thread name: e.g. “worker[0-4]”, or * to apply to all available threads. * for the process set will match all ranks.

When focused on a threaded process set, operations act similarly to using thread apply on that set of threads (see help thread).

delete

Summary: Delete breakpoints and other items.
Usage: delete <breakpoint id>
delete checkpoint <checkpoint id>
delete $decomposition
delete $assertion_script
Shortcut: d

Arguments: The name or id number of the item to delete.

For breakpoints and checkpoints:
Delete the breakpoint or checkpoint specified by the id argument in
every application rank in the current process set. If no argument is
supplied, all breakpoints in the current process set are deleted.

For decompositions and assertion scripts:
Delete the decomposition or assertion script with the given $name.

directmode

Summary: Direct gdb mode command.
Usage: gdbmode [gdb command]
Example command: gdbmode
Example command: gdb x/4xb rawPtr

Arguments: None.

Description:
This function gives direct access to the gdb instances on the ranks, either to access functionality not exposed by gdb4hpc, to view data like pointer values that are normally hidden by gdb4hpc, or possibly to view data structures gdb4hpc doesn’t understand.

Some care should be taken since some operations like changing breakpoints or continuing could compromise the state of gdb4hpc.

When called with arguments, it executes that command directly in gdb and prints the results aggregated by ranks.

When called with no arguments, this command puts the debugger into a gdb direct mode, and will send all subsequent commands directly to gdb until it receives the command “end”. “quit” is also available which will quit gdb4hpc.

disable

Summary: Disable breakpoint command.
Usage: disable [<breakpoints>]
Example command: disable 1

Arguments: breakpoints (list of breakpoint numbers)

Description:
Disable the list of breakpoints specified by the breakpoints argument in every application rank in the current process set. If no argument is supplied, all breakpoints in the current process set are disabled.

down

Summary: Move stack frame down command.
Usage: down [<num>]
Example command: down

Arguments: num (number of frames to go down)

Description:
Move the call stack down by num frames. If no argument is supplied, then move down by one frame.

enable

Summary: Enable breakpoint command.
Usage: enable [-delete|-once] [<breakpoints>]
Example command: enable 1

Arguments: breakpoints (list of breakpoint numbers)

Description:
Enable the list of breakpoints specified by the breakpoints argument in every application rank of the current process set. If no argument is supplied, all breakpoints in the current process set are enabled. This command is used to cancel the effect of the “disable” command. Valid option arguments to this command are -delete and -once. Only one argument can be provided. The -delete argument will cause the breakpoint to be deleted when it is hit. The -once argument will cause the breakpoint to be disabled when it is hit.

ignore

Summary: ignore next n triggers of a breakpoint.
Usage: ignore <breakpoint> <count>
Example command: ignore 1 100

Ignore the next count occurrences of breakpoint <breakpoint>.
Using count 0 will stop the program the next time the breakpoint
is hit.

interpreter exec

Summary: Execute a command under a specific interpreter.
Usage: interpreter-exec <interpreter> <command>
Example command: interpreter-exec mi -exec-next

Execute a command under a specific interpreter. gdb4hpc starts with
the “console” interpreter by default, but supports others.
Each interpreter has different commands and different output formats.

Available interpreters are “console” and “mi”.

condition

Summary: stop at a breakpoint only if a condition is met.
Usage: condition <breakpoint> [condition]
Example command: condition 1 i==4

Set a condition for a breakpoint. The condition is evaluated
each time the breakpoint location is reached, and the program is
stopped only if the condition evaluates to true or non-zero.
The conditions is evaluated on the back end so procset scoping
is not allowed, e.g. $a::i == 4.
Calling with no condition makes the breakpoint unconditional.

finish

Summary: Finish function execution command.
Usage: finish
Example command: finish

Arguments: None.

Description:
Execute every rank in the current process set from its current location until the selected stack frame returns.

focus

Summary: Focus process set command.
Usage: focus <proc_set>
Example command: focus $all

Arguments: proc_set (named process set)

Description:
Change the focus of the current process set to the named process set specified by the argument proc_set. This can be the app_handle of an attached/launched application or a process set created with the defset command. A temporary proc_set will be created if the user focuses explicitly on a subset of ranks using the process set notation. In that case, the temporary proc_set will be removed when the focus is changed a second time.

For example, if the app_handle $a{1024} was provided to launch an application, focus $a{0..32,64} would create a temporary process set named a_temp containing ranks zero thru thirty-two and rank sixty-four of application $a. The temporary process set name cannot be subsequently used in process targeted commands” in the same way a defined process set can.

The process set expression can optionally contain a thread set after a colon, for example defset $thread_set $a{0:2} will focus all actions onto thread 2 within rank 0. The thread expression is a comma separated list of thread ids, ranges of thread ids as <min-index>..<max-index>, and “<pattern>” for regular expressions that match on the thread name: e.g. “worker[0-4]”, or * to apply to all available threads. * for the process set will match all ranks.

When focused on a threaded process set, operations act similarly to using thread apply on that set of threads (see help thread).

frame

Summary: Print selected stack frame command.
Usage: frame [-v]
Example command: frame

Arguments: optional -v for “verbose”.

Description:
Display the selected stack frame from every application rank in the current process set. -v (verbose) prints the program counter address for the frame.

halt

Summary: Halt all executing processes.
Usage: halt [-a]
Example command: halt

Arguments: None.

Description:
Halt all executing ranks in the current process set. This is equivalent to sending a SIGINT to the ranks. In non-stop mode only the current thread is halted; use halt -a to stop all threads.

help

Summary: Help command.
Usage: help [<command>]
Example command: help
Shortcut: h

Arguments: command (command specific message)

Description:
Display help information about commands. If no argument is given, display a list of all valid commands.

launch

Summary: Launch an application.
Usage: launch <app_handle> <application> [application args]
[–args=”<args>” OR -a “<args>”]
[–launcher=”<launcher_name>” OR -l “<launcher_name>”]
[–launcher-args=”<launcher_args>” OR -g “<launcher_args>”]
[–launcher-input=<path_to_file> OR -i <path_to_file>]
[–workdir=<path> OR -d<path>]
[–env=”<name=value>”]
[–qsub=<batch_template> OR -q <batch_template>]
[–sbatch=<batch_template> OR -s <batch_template>]
[–gpu]
[–gdb=<gdb_app>]
[–gdb-data-dir=<datadir>]
[–non-mpi]
[–debug[=<path>]]

Example command: launch $a{20000} ./helloworld ./hello_world_config.txt

Arguments: app_handle (debugger variable syntax),
application (name of application), args (application arguments),
launcher_args (application launcher arguments),
path_to_file (application launcher stdin file), path (cwd path),
name=value (environment string), batch_template (job script)

Description:
Start an application with process set handle of app_handle and optional application arguments args, application launcher arguments launcher_args, application launcher input file path_to_file, and application environment string name=value via the system application launcher. Integration with qsub and sbatch is supported. See below for more information on use.

The app_handle argument must be a debugger variable array specifying the number of ranks in the application. For example, an application with an app_handle of $a{9} will launch the appication with 9 application ranks. The application launcher is system dependent and is automatically determined. The aprun and srun application launchers are supported.

If you wish to enable GPU debugging, add –gpu to your launch command. This will enable the use of a GPU supported version of gdb and set several environment variables for debugging.

–gdb is an experimental option to use a specific debugger with each process, for example: –gdb=/opt/rocm-3.9.0/bin/rocgdb

–gdb-data-dir is an experimental option to use a specific data directory for –gdb

The optional args, launcher_args, and name=value arguments should be enclosed with quotes. The args argument is used to pass arguments directly to the application. The launcher_args argument is used to pass additional arguments specific to the application launcher (either aprun or srun). The path_to_file argument is used to redirect the stdin of the launcher process to a file. The path argument is used to change the current working directory of the application launcher. The path can be a full path or relative path. If path is omitted, then the current working directory of the application launcher will be changed to the location where the application a.out is located. The name=value argument is used to set environment variables for the application launcher. The –env argument can be used more than once to set multiple environment variables.

For example: To provide the aprun arguments -N1 -L134 you would use –launcher-args=”-N1 -L134” Alternatively you can use the short option of -g “-N1 -L134”

Note that if using the short option -d, the argument is optional and a space should not be placed between the option and the argument (i.e. use -d<path>).

Workload manager support:

Launching on systems whose default workload manager is not SLURM or ALPS requires passwordless (public key) ssh to the compute nodes of active jobs. Contact a system administrator or the system usage guide for details on whether this is supported and how to set this up.

To launch an application using a non default launcher, the –launcher argument can be used. This can be the name of an executable reachable in PATH or an absolute path. Setting this option only affects the name of the launcher executable itself and does not change the launch system. To launch with a non default launcher that is not related to the default running workload manager (such as mpiexec.hydra on a slurm system), this variable should be used in conjunction with setting CRAY_CTI_WLM to “generic” to ensure the launch mechanisms work with the specified launcher. Note that setting CRAY_CTI_WLM to “generic” requires passwordless SSH to the compute nodes. Specify the name of the launcher to use as the argument to the launcher option.

For example, launch $a{10} –launcher=”myrun”

To launch inside of a job reservation, the –qsub or –sbatch arguments can be used. Modify the existing job script for the application by placing the comments #cray_debug_start and #cray_debug_end around the aprun/srun command to be debugged.

For example, we would modify an existing job script named sample.pbs:

#cray_debug_start aprun -n128 -N32 a.out #cray_debug_end

Everything between the comment lines will be ignored. The user is still required to recreate the aprun/srun command using the existing launch command arguments.

Using the above, the user would launch with the following arguments: launch $a{128} –launcher-args=”-N32” –qsub=sample.pbs a.out

–non-mpi is an experimental launch method for handling cases where the WLM launch barrier is not available, typically for non MPI applications.

–debug writes gdb4hpc logging information to a directory. If not specified via =<logdir>, the enviroment variable $CRAY_DBG_LOG_DIR is used. The directory needs to be a location writable from the compute nodes.

When using the Slurm workload manager, GDB4hpc supports launching heterogeneous MPMD jobs using syntax similar to Slurm’s native MPMD launch, with job steps separated by a colon :. The rank count for each individual step must be supplied via –launcher-args, and sum to the ranks specified in the process set definition. For example: launch $a{5} –launcher-args=”-n3” a.out –args=”a_arg” : –launcher-args=”-n2” b.out –args=”b_arg”

srun

Summary: Launch an application with srun.
Usage: srun [OPTIONS(0)… [executable(0) [args(0)…]]] [ : [OPTIONS(N)…]] executable(N) [args(N)…]
Example command: srun -n3 -N6 -p allnodes ./app

Description:
Launch an application using the same syntax as the srun command.

gdb4hpc will parse the srun command and internally create a launch command with equivalent options. Not all srun options are supported. To launch an application with an unsupported option, use the launch command with --launcher-args, which forwards all options verbatim to srun without any intermediary parsing step.

All srun options with a short (one letter) version are supported, along with their corresponding long versions. Additionally, the following long options are supported:

--ntasks-per-node
--exclusive
--gres

run

Summary: Start debugged program.
Usages:
launch an application: run $<procset>{<num-ranks>} <application> [launch arguments]
attach to a running application: run $<procset> <job-id> [attach arguments]
relaunch the current procset: run
relaunch a specific procset: run $<procset>
launch a clone of a procset: run $<current-procset> $<clone-procset>
Example commands:
run $a{2000} ./helloworld –launcher-args=”–cpus-per-task=2”
run $a 255524.0 –gpu
run
run $a
run $a $b

Description:
For the launch and attach case, run is a simple alias. See “help launch” and “help attach” for the full description. For the relaunch usage, run kills the current run and does a new launch with the same application, arguments, and breakpoints; and stops it at the beginning of the execution. The clone case does the same relaunch, but creates a new procset handle while leaving the existing launch unchanged.

kill

Summary: Send signal to attached/launched application.
Usage: kill <app_handle> [signal]
Example command: kill $a SIGCONT

Arguments: app_handle (process set handle), signal (name or number)

Description:
Send signal to previously attached/launched application specified by app_handle.

If no signal is specified, The entire application will be killed with SIGKILL, even if individual ranks are specified in the app_handle.

list

Summary: List specified function or line command.
Usage: list [-] [<expr>]
Example command: list 20
Shortcut: l

Arguments: - (backward direction), expr (list expression syntax)

List expression syntax

The list expression can contain an argument of the following forms:

linenum - display source lines around specified line number.
file:linenum - display source lines around specified line number in specified source file.
function - display source lines around specified function.
file:function - display source lines around specified function in specified source file.
*addr - display source lines around the line corresponding to address addr.

Additionally, a comma separated list of two arguments can be defined in expr to display all source lines between arg1 and arg2.

Description:
Display source code of the ranks in the current process set. With no argument, ten lines of code are displayed from the previous listing. If - is specified, ten previous lines of code are displayed.

next

Summary: Next command.
Usage: next [<num>]
Example command: next
Shortcut: n

Arguments: num (number of lines to advance)

Description:
Step program, but do not follow into subroutine calls. If the argument num is supplied, advance num lines.

ompd

Summary: OpenMP debugging functions.
Usage: ompd [subcommand]

Whether the ompd function is a available and which sub-functions
are available is installation specific. Use
“ompd help” to get the commands available with the current process.

pipe

Summary: Pipe the output of a gdb command to a shell command.
Usage: pipe <command> | <shell command>
Usage: pipe -d <delim> command <delim> <shell command>
Example command: pipe p/1 my_array | grep 0x0

Description:
Pipes the result of any gdb command to a shell command. Error output from the command is not redirected. The -d version allows you to use a character sequence other than “|” for redirection.

print

Summary: Print the value of an expression.
Usage: print [/format] <expr>
Example command: print prog_variable
: p/x val
: p/csv array
Shortcut: p

Description:
Print the value of expr for each application rank in the current process set. The expr can contain any combination of debugger convenience or application variables. Valid application variables are those of the lexical environment of each ranks selected stack frame, plus all those whose scope is global or over an entire file.

Application ranks can be targeted using the process set notation for the desired app_handle. For example, to print variable myVar in ranks 1,3,4,5,10 of application $a, issue: print $a{1,3..5,10}::myVar.

Format strings change the style of printing: /x for hexadecimal, /o for octal, /t for binary (“two”) and /z for zero padded binary. gdb4hpc extends the gdb formats with /1 to print one element per line, and /csv to print arrays as comma separated values.

ptype

Summary: print the definition of a type.
Usage: ptype [/m|/t|/mt] <type-or-variable>
Example command: ptype timeval
Example command: ptype /m x

Print the type information for either a type or a program variable.
The optional format options:
/m suppresses printing member functions.
/t suppresses printing typedefs.
/mt suppresses both.

python

Summary: enter the integrated python interpreter.
Usage: python

This enters a python interpreter, and all lines enterred will be
interpreted as python code.

show python

Summary: set/show if gdb4hpc is in python mode
Usage: set python <on|off|auto>
show python

In python mode, gdb4hpc will use python versions of certain commands. For
example, normally the backtrace command prints the C/C++/Fortran backtrace,
but in python mode it prints the backtrace of the interpreted python script
rather than the backtrace of the cpython interpreter itself.

The following options are available:

on: use python mode for all future launches.
off: don’t use python mode for any future launches.
auto: automatically turn on python mode when python is detected.

The default setting is auto.

In addition to affecting future launches, setting python mode directly
affects currently launched procsets. For example, if you really wanted to see
the C backtrace of the cpython interpreter, you could turn python mode off
and then use the backtrace command. Setting python mode to auto has no effect
on running procsets.

gdb4hpc supports the following python versions:
3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13

quit

Summary: Quit command.
Usage: quit
Example command: quit
Shortcut: q

Arguments: None.

Description:
Quit the debugger.

release

Summary: Release attached/launched application from debugger control.
Usage: release <app_handle>
Example command: release $a

Arguments: app_handle (process set handle)

Description:
Release previously attached/launched application specified by app_handle.

Every rank of the app_handle will be released, even if individual ranks are specified.

restart

Summary: Restore program to state of previously created checkpoint.
Usage: restart <checkpoint id>
Example command: restart 2

Description:
For each rank linked to the specified checkpoint, restore the program to one previously forked and set aside by the checkpoint command.

set variable

Summary: Evaluate an expression and assign the result to a variable.
Usage: set variable <name> = <expression>
set <name> = <expression>
Example commands: set program_variable_foo = program_variables_bar
set $client_value = program_variable

Description:
The set variable command evaluates the given expression and assigns the result to a variable.

<name> may be a program variable or the name of a client side value. To assign to a client side value, begin the <name> with $. <expression> may be any expression valid in the current language.

The set command is an alias for set variable if the given argument doesn’t correspond to a debugger setting.

shell

Summary: Launch a shell command locally
Usage: shell [-r] [command]
Example: shell -r hostname

Description:
Run a shell command on the local host and print out the result.

The -r (remote) option runs the shell command on every rank currently in focus.

With no arguments, start a new subshell or re-enter a suspended (local) subshell. All entries are sent to the subshell until either the “exit” or “suspend” command is entered.

source

Summary: Read and execute debugger commands.
Usage: source [--echo|--no-echo|--keep-going|--no-keep-going]* <filename>
Example commands: source myfile
source --echo --keep-going myfile

Arguments: filename (path to named command file)
–echo/–no-echo (enable or disable echoing)
–keep-going/–no-keep-going (change behavior on error)

Description:
Read debugger commands from a file to automate debugging tasks. The –echo and –keep-going flags change the behavior of gdb4hpc while running the commands. –echo tells gdb4hpc to print out each command it executes along with the line number it corresponds to in the sourced file. –keep-going tells gdb4hpc to continue executing the contents of a file, even if a command fails. If no flags are specified, the behavior is inherited from the current environment.

step

Summary: Step command.
Usage: step [<num>]
Example command: step
Shortcut: s

Arguments: num (number of lines to step)

Description:
Advance the program to the next source line, following into subroutine calls. If the argument num is supplied, then advance num lines.

tbreak

Summary: Set temporary breakpoint command.
Usage: tbreak <location>
Usage: tbreak <location> thread <thread-id>
Example command: tbreak main.c:20
Example command: tbreak myFunction thread 2

Arguments: location (line number, function name, source-file:line or address)

Description:
Set a temporary breakpoint in every application rank in the current process set. The breakpoint will be deleted after being hit. This is equivalent to issuing a “break” followed by “enable -delete” with the breakpoint number.

The location of the breakpoint can be defined as a line number, a function name, or * and an address. If an address is specified, it must begin with an asterisk (*). For example, break *0x0000000000400693 would break at address 0x400693. Adding thread thread-id create a thread specific temporary breakpoint that will only stop when that thread hits the break point.

thread

Summary: Switch between threads.
Usage: thread <thread-id>
Example command: thread 5

Arguments: thread-id

Description:
Change the thread context in every application rank in the current process set.
Subsequent commands like “print”, “bt”, “next” and “step” will be
directed to the thread-id

Summary: apply a command to a set of threads.
Usage: thread apply <thread-list> | all [-ascending] [-s] [-q] [-c] [-v] <command>
Example command: thread apply all -s p i
Example command: thread apply 1 2-4 9..11 “worker 1.[0-9]” bt 1
Example command: thread apply 1,2-4,9..11,”worker 1.[0-9]” info frame

Description:
Applies command to every thread in the selection list with the following flags
to control the output:
-q (quiet) disables printing the thread information for each result
-s (silent) disables output from threads where the command generates an erro
(e.g. variable not defined).
-v (verbose) suppresses stripping out LWP and pointer values from the output.
Especially when not combined with -q this will tend to produce a full result for
each member of the procset.
-c and -ascending (continue after error) and list in ascending order are the default behaviors
and are included for compatibility with gdb commands.
<thread-list> accepts the same syntax as info threads
<command> is any valid gdb command

Summary: apply a command silently to all threads.
Usage: taas [-q] [-c] [-v] <command>

Description:
taas is a shortcut for thread apply all -s

Summary: Name the current thread.
Usage: thread name [<name>]
Example command: thread name main

Description:
Renames the current thread to name. If no name argument is given, remove any previous name assigned
via thread name and revert to the program’s name for the thread.

Summary: Find threads using a regular expression
Usage: thread find <regex>
Example: thread find collector

Description:
Prints threads whose full id contains a regular expression. The match is against the full
thread identifier shown by “info threads -v” which may include not just the number and name
but also the thread’s address and LWP id

up

Summary: Move stack frame up command.
Usage: up [<num>]
Example command: up

Arguments: num (number of frames to move up)

Description:
Move the call stack up by num frames. If no argument is supplied, then move up by one frame.

viewset

Summary: View process set command.
Usage: viewset [<proc_set>]
Example command: viewset

Arguments: proc_set (named process set)

Description:
Display information about defined process sets. With no argument provided, information about all defined process sets is displayed.

watch

Summary: Set watchpoint command.
Usage: watch [-a|-w|-r] <expression> [thread <thread-id>]
Example command: watch -a foo(1,1)
Example command: watch i thread 4

Arguments: expression (variable expression)

Description:
Set a watchpoint in every application rank in the current process set. The expression of the watchpoint can be defined as a variable, an address, a variable operation (such as a cast), and other arbitrarily complex expressions using any operators valid in the program’s native language.

Valid option arguments to this command are -a and -w. The -a option specifies that an access watchpoint should be created. Access watchpoints will cause the program to break when the address defined by the expression is read or written to. To set a watchpoint on an address, use an expression of the form *0xXXXXXX with the -a option. The -w option specifies that a write watchpoint should be created. Write watchpoints will cause the program to break when the address defined by the expression is written to. The -r option specifies that a read watchpoint should be created. Read watchpoints will cause the program to break when the address defined by the expression is read from. If no argument is provided the watchpoint will default to a write watchpoint.

If the expression corresponds to a single address, a hardware watchpoint will be created. If the expression corresponds to a range of addresses a software watchpoint will be created. Software watchpoints cause application performance to slowdown by orders of magnitude and should be avoided, if possible.

If the variable defined by the expression goes out of scope, the watchpoint will cause the application to break and the watchpoint will be removed.

Adding thread thread-id creates the watchpoint that only applies to the given thread.

catch

Description:
Set a catchpoint on a specified event. Available events are:

throw: catch the throwing of a C++ exception
catch: stop when an exception is caught
fork: catch calls to fork
vfork: catch calls to vfork
exec: catch calls to exec
load <regex>: stop when a shared library matching regex is loaded
unload <regex>: stop when a shared library matching regex is unloaded

For each event, the process is stopped just after the event occurs.

checkpoint

Summary: Fork program for restoration at a later point.
Usage: checkpoint
Example command: checkpoint

Description:
Fork all ranks in focus and set aside the new processes for restoration later with the restart command.

Delete a checkpoint with the delete checkpoint command.

whatis

Summary: Whatis data type of expression.
Usage: whatis [-type] <expr>
Example command: whatis prog_var

Arguments: expr (expression syntax)

The expr argument can be the name of a program variable in a launched
application or a type name defined by typedef, or “struct STRUCT-TAG,”
or “union UNION-TAG,” or “enum ENUM-TAG.”

Description:
Display the data type of an expression for each application rank in the current process set. By default, only the base type is displayed. If the -type argument is provided, the expr is interpreted as being a defined type name, struct, union, or enum. A type is always displayed as a full type definition.

decomposition

Summary: Decomposition creation command.
Usage: decomposition $<name> [dim_def1] [,dim_def2] [,dim_def3]…
Usage: decomposition $<name>
Example command: decomposition $myName 100, 1000/10

Description:
This function defines an n dimensional array that can treat data on multiple ranks as a single unified array.

In simple form: decomp $d 10, 12 Which can be used to print an basic double* my_array as a 10 x 12 array: p $d{my_array}

The decomposition can also distribute an array over multiple ranks. For example, when running 10 ranks: decomp $d 100/10, 12 “p $d{my_array}” will print a 100 x 12 array where: rank 0 provides elements [0..9][0..11], rank 1 elements [10..19][0..11], etc.

Several forms of decomposition are supported: Block: num_elems / num_ranks : the array is broken into equal sized chunks Cyclic: num_elems % num_ranks : the array is assigned in round robin so for 100 % 10, elements 0,10,20..90 are on rank 0, 1,11,21.. are on rank1 Block-cyclic: num_elems / num_ranks % block_size combines both in elements are distributed round robin in block_size chunks, so: in 100/10%5, elements 0..4 and 50..54 are on rank 0, elements 5..9 and 55..59 are on rank 1, etc.

Arrays can be distributed over multiple dimensions, which means the ranks also form a grid. By default the rank ordering uses the natural ordering of the language, leftmost major in C/C++, rightmost major in fortran. This can be overridden using “order(n)”, thus to reverse the process grid in C use: decomp $d 100/10 order(1), 120/12 order(2). in this case rank 0 has [0..9][0..9], rank 1 has [10..19][0..9] and rank 35 has [50..59][30..39]. Note that “order” only affects the assignment order of the ranks, it cannot transpose an array.

See the decomposition tutorial in this package for more examples.

If the command is called with only a decomposition name, the command enters a decomposition definition mode where only decomposition specialized functions are available. Use the “help” command while in decomposition more for details.

save decomp

Summary: Save decomposition state to a file.
Usage: save decompositions <filename>
save decomposition $name <filename>

Saves decomposition(s) to a file on disk. Use the source command to restore them.

The saved file contains plaintext gdb4hpc commands and can be edited with any
text editor.

build

Summary: Assertion script creation command.
Usage: build <name>
Example command: build $my_script

Script mode commands: assert, end, help, set, show
Arguments: name (debugger variable)

Description:
Begin creation of a new assertion script. Build sub-commands are read until “end” is encountered.

start

Summary: Assertion script execution command.
Usage: start <name>
Example command: start $my_script

Arguments: name (built assertion script name)

Description:
Begin execution of a built assertion script. Assertion script execution continues until assertion failures (if set stop is used), the end of the program is reached, or a deadlock is detected. Stopping behavior can be changed via the “set stop” command.

stop

Summary: Assertion script halt command.
Usage: stop
Example command: stop

Arguments: None.

Description:
Halt the current assertion script execution immediately. This command is used to abort assertion script execution.

info

Summary: Info command.
Usage: info [<subcommand>]
Example command: info

Arguments: subcommand (info subcommand)

Description:
Display information relating to the application being debugged. If no argument is given, display a list of all valid info subcommands.

maint

Summary: Maint command.
Usage: maint [<subcommand>]
Example command: maint

Arguments: subcommand (maint subcommand)

Description:
Commands that are useful only to maintainers or are experimental. If no argument is given, display a list of all valid maint subcommands.

maint set

Summary: Maint set command.
Usage: maint set [<subcommand>]
Example command: maint set

Arguments: subcommand (maint set subcommand)

Description:
Set information relating to environment settings and behavior that is useful only to maintainers or is experimental. If no argument is given, display a list of all valid maint set subcommands.

maint show

Summary: Maint show command.
Usage: maint show [<subcommand>]
Example command: maint show

Arguments: subcommand (maint show subcommand)

Description:
Show information relating to environment settings and behavior that is useful only to maintainers or is experimental. If no argument is given, display a list of all valid maint show subcommands.

set

Summary: Set command.
Usage: set [<subcommand>]
Example command: set

Arguments: subcommand (set subcommand)

Description:
Set information relating to environment settings and behavior. If no argument is given, display a list of all valid set subcommands.

If an argument is given that does not match a subcommand, set is an alias for set variable.

set print

Summary: Set print command.
Usage: set print [<subcommand>]
Example command: set print

Arguments: subcommand (set print subcommand)

Description:
Settings relating to the way things are printed. If no argument is given, display a list of all valid set print subcommands.

set break

Summary: Set break command.
Usage: set break [<subcommand>]
Example command: set break

Arguments: subcommand (set break subcommand)

List of set breakpoint subcommands:
set breakpoint pending – Set pending breakpoint behavior (on, off, auto)

Description:
Settings relating to breakpoints. If no argument is given, display a list of all valid set break subcommands: on, off, or auto. Default setting is “auto”, which will query to create a pending breakpoint. If set to “on”, a pending breakpoint will automatically be created. If set to “off”, attempting to set a pending breakpoint will produce an error.

show

Summary: Show command.
Usage: show [<subcommand>]
Example command: show

Arguments: subcommand (show subcommand)

Description:
Show information relating to environment settings and behavior. If no argument is given, display a list of all valid show subcommands.

save

Save debugger state to a file. Use the source command to load the state.

save all

Summary: Save all debugger state to a file.
Usage: save all <filename>

Saves all breakpoints, decompositions, and assertion scripts to a file on disk.
Use the source command to restore them.

The saved file contains plaintext gdb4hpc commands and can be edited with any
text editor.

show print

Summary: Show print command.
Usage: show print [<subcommand>]
Example command: show print

Arguments: subcommand (show print subcommand)

Description:
Show information relating to the way things are printed. If no argument is given, display a list of all valid show print subcommands.

show break

Summary: Show break command.
Usage: show break [<subcommand>]
Example command: show break

Arguments: subcommand (show break subcommand)

Description:
Settings relating to breakpoints. If no argument is given, display a list of all valid show break subcommands.

unset

Summary: Unset command.
Usage: unset [<subcommand>]
Example command: unset

Arguments: subcommand (unset subcommand)

Description:
Unset information relating to environment settings and behavior that was previously set. If no argument is given, display a list of all valid set subcommands.

usage

Summary: Usage command.
Usage: usage [<subcommand>]
Example command: usage

Arguments: subcommand (usage subcommand)

Description:
Display information relating to script mode commands and their use. If no argument is given, display a list of all valid usage subcommands.

info breakpoints

Summary: Breakpoint information.

Description:
Display the status of user defined breakpoints.

The type column indicates one of the following:

breakpoint - normal breakpoint
watchpoint - watchpoint

The disp column contains keep, del, or dis to indicate the disposition of the breakpoint after it is hit. If disp is displayed as dis, this means that the breakpoint will be disabled. The address and what columns indicate the address and file/line number of the breakpoint respectively.

info checkpoints

Summary: Checkpoint information.

Description:
Display current checkpoints. Columns display the following:

Checkpoint: The checkpoint id for use in restart and delete commands
Procset: The ranks where this checkpoint exists. When the restart command is used, all of these ranks will be affected.
Location: Location in program where ranks will be after restarting at this checkpoint.

info locals

Summary: Local symbol information.

Description:
Display a list of the local symbols in the current stack frame for each application rank in the current process set.

info args

Summary: Argument variables of the current frame.

Description:
Display a list of the arguments of the current stack frame for each application rank in the current process set.

info application

Summary: Application information.

Description:
Display information related to currently controlled applications. The handle column specifies the process set handle used to identify the launched/attached application. The ranks column indicates the number of application ranks. The focus column indicates if this application’s process set is currently focused upon. The command column identifies the command that was passed to the application launcher in order to launch the application.

info cuda

Summary: cuda specific information.
Usage: info cuda [subcommand]
Example: info cuda threads
Description:
Display cuda specific information. This will only be available when a cuda application is launched or attached with the –gpu flag. Use “cuda help info” for the available subcommands.

info scope

Summary: Local symbol information relative to a scope.
Usage: info scope <location>
Example command: info scope source.f:20

Arguments: location (function, line, or addr)

Description:
Display a list of the local symbols relative to a scope for the application. The provided argument can either be a function name, a line number in a source file, or an address prefixed with *.

info source

Display the current source file.

info sources

Summary: Source file information.

Description:
Display the source files in the application. This command only displays the source files that have been located and are available to be used in conjunction with the list command. Try “set directory” to improve the results.

info status

Summary: Status command.”, “
Description:”, “Display the status of lauched/attached applications.

info threads

Summary: List threads.”, “
Usage: info threads
Usage: info threads 2
Usage: info threads 1-4 6
Usage: info threads 2,4..8
Usage: info threads “worker[0-9]”
Arguments: [-v] [-l] optional list of thread ids”, “
Description:
List the known threads in the current process set. -v option will suppress the removal of the address and LWP for each thead (which will usually create a full thread set output for each process). -l prints every thread on a separate line instead of collecting threads at the same location. With no thread ids, this command displays all known threads. A subset of threads to display can be specified with a space or comma separated list of thread ids with ranges, e.g.: 2 4 7-10 or 2,4,7..10 The list may also contain regular expressions within quotes, e.g "thread.*1" or 'thread.*1' which will match the set of threads returned by thread find.

info decomposition

Summary: Display defined decompositions.”, “
Description:”, “Display the names of defined decompositions.

info diff

Summary: Display saved assertion script difference delta information.
Usage: info diff”
info diff <$script>
info diff <$script> <assert id>
Description:
Display the set of saved difference data collected during assertion script execution. Without a script $name argument, the command will show a list of all collected diffs from all assertion scripts. To view a list of collected diffs for a specific assertion script only, pass its $name as the third argument. To see the actual difference value, pass the $name and the id of the assertion you want to view the differences of.

info script

Summary: Assertion script information.”
Usage: info script
: info script <$script>
: info script <$script> results
Description:
Display information about saved assertion scripts and their results. Without a script $name argument, the command will print a list of all assertion scripts and indicate if one of them is currently running. Passing a $name argument will print the actual assertions within the script. Passing an additional “results” argument will show a detailed view of the results of the assertion script.

info iostat

Summary: Display a table of Fortran Runtime Errors
Usage: info iostat
: info iostat <code>
Description:
Display information about the Fortran Runtime Errors. Without providing a code argument, the command will print a list of all Fortran Runtime Error codes from cce. Passing a code argument will print the detailed information for the given code.

maint echo

Summary: Echo command.
Usage: maint echo <string>
Example command: maint echo print this string

Arguments: string (string to echo)

Description:
Echo a string as output.

maint sleep

Summary: Sleep command.
Usage: maint sleep [<secs> [-r]]
Example commands: maint sleep 5
maint sleep 5 -r

Arguments: secs (seconds to sleep)
-r (remote sleep)

Description:
Pause the debugger for secs seconds. If secs is omitted, the debugger will sleep for a default of five seconds. If -r is specified, tell the remote debugger servers to sleep instead of the client.

maint direct

Summary: Passthru command.
Usage: maint passthru <command>
Example command: maint passthru show version

Arguments: command (gdb command)

Description:
Pass a command directly to the debug backends of every application rank in the current process set. This can be used to talk directly to gdb. Console output will be aggregated and returned.

maint system

Summary: System command.
Usage: maint system <command>
Example command: maint system ls -la

Arguments: command (shell command)

Description:
Run a command in the shell environment of the client. This can be used to enumerate information about the system the client is running on. The stderr of the command will be redirected to stdout and printed in the result.

set cuda

Summary: set cuda specific settings.
Usage: set cuda [subcommand]
Example: set cuda hide_internal_frame on
Description:
Set cuda-gdb specific settings. This will only be available when a cuda application is launched or attached with the –gpu flag. Use with no arguments for the available subcommands.

directory

Summary: Add to the source Directory search path.
Usage: dir <path> [<path>]..
Usage: dir
Example command: dir /path/to/source

Arguments: path (directory containing source)

Description:
Add the directory defined by path to the beginning of the search path for the application source files that were used to generate symbol information for application. Typically this is used when an application was compiled on a different machine than run, but the source files used to compile the application are available at a new location. This will cause cached information about source file locations and line positions to be lost.

With no arguments, reset the search path to $cdir:$cwd

set directories

Summary: Replace the directory search source path.
Usage: set directories [<path>]..
Example command: set directories /path/to/source1 /path/to/source2

Arguments: paths (directory containing source)

Description:
This is like the directory command except the previous path is removed.

set substitute path

Summary: Define a rule to replace a compile path with a source path.
Usage: set substitute-path <from-path> <to-path>
Example command: set subtitute-path /orig/dev /mnt/cross

Description:
Add a source path substitution rule, which is useful when an application was built somewhere else. Unlike the directory command, it maps the entire build tree to a new location.

show substitute path

Summary: Show a rule to replace a compile path with a source path.
Usage: show substitute-path [from-path

Description:
Show the source path substition rules. If an path argument is given, show just any rule mapping that path.

unset substitute path

Summary: Unset one or all compile path to source path rules.
Usage: unset substitute-path [from-path

Description:
Unset the source path substition rules. If an path argument is given, unset only any rule mapping that path.

set error

Summary: Warn and error epsilon for floating point comparison.
Usage: set error <warn limit> <error limit>
Example command: set error 1e-16 2e-15

Default setting: ulp ulp
Where ulp is the smallest non-zero double precision floating
point value of the current machine.
Arguments: warn limit (float), error limit (float)

Description:
Set the tolerance level for compare/assert commands difference deltas when comparing floating point values. Differences with deltas below the warning epsilon are ignored. Differences with deltas between the warning and error epsilon will be displayed using the current display mode setting and reported as a warning. Any differences with deltas above the error epsilon will be displayed using the current display mode setting and reported as an error. Error differences will halt the assertion script execution if stop mode is true.

set environment

Summary: Set environment variable value.
Usage: set environment <name> <value>
Example command: set environment var val

Arguments: name (environment variable name), value (value to assign to
environment variable)

Description:
Set the environment variable named name to the value provided by the value argument. Arguments can optionally be enclosed with quotes, the actual quotes will be stripped out for the final assignment. This will not affect any previously launched applications and will only take effect upon the next launch command. All environment variables passed this way are put into the forked environment of the application launcher process (aprun/srun).

set logging

Summary: Set output file for logging client output.
Usage: set logging <subcmd> <value>
Example command: set logging enabled on

Description: Used to enable and control logging functionality of debug client.
Can enable logging to file and define or change file location. See following lines for valid subcmds and arguments:

set logging enabled [on | off]: Enables or disables logging functionality, defaults to “on” if no argument passed. set logging file <file>: Sets filepath of file to write to, defaults to “gdb4hpc.txt” if no argument pased. set logging overwrite [on | off]: Enable overwrite to have logs overwrite file instead of appending to it set logging redirect [on | off]: If redirect enabled, output will only go to log file instead of log file and terminal.

set function

Summary: Set function definition.
Usage: set function <function_name>(<function_args>) = <expr>
Example command: set function myfunc(myvar) = 1

Arguments: function_name (name of function), function_args (function
arguments), expr (function expression definition)

Description:
Define a function macro named function_name. The expression expr can contain format arguments that are substituted by actual arguments supplied to the function. Recursive function expression calls are not allowed.

set scheduler locking

Summary: Set the behavior of the thread schedule locking during execution
Usage set scheduler-locking off|on|step|replay
Example command: set scheduler-locking off

Description:
Set the mode for the locking scheduler:
off : no locking, threads may preemt any time
on : full locking, no thread except the current thread may run
step : scheduler is locked during stepping commands (step,next,stepi,nexti)
replay : scheduler is locked during replay mode and unlocked during normal execution.

set non stop

Summary: Set the behavior of the threads during a break
Usage set non-stop off|on
Example command: set non-stop on

Description:
Set the non-stop mode
off : stopping one thread in a process stops all threads
on : other threads continue running when one is stopped

set subcmd args help

Summary: Set or clear arguments to use for new processes
after the “run” commmand is invoked.
Usage: set args <arg1> <arg2> “<multi> <word> <arg3>”
Example command: set args -x filename -w “mutli word”
-> argv = {“-x”, “filename”, “-w”, “multi word”}
Example command: set args
-> argv = {}

Description:
Set new arguments to give to a new job launch when the “run” command is used. To set arguments for only a certain procset, use the “focus” command to focus on the procset first. If no arguments are passed, the stored list of arguments is reset.

set print entryframe

Summary: Set printing of frame on entry mode.
Usage: set print entry-frame [true|false]
Example command: set print entry-frame false

Default setting: true
Arguments: true or false

Description:
Determines if the frames of the application ranks are printed on launch/attach.

set print sourceonstep

Summary: Set printing of source on step.
Usage: set print source-on-step [true|false]
Example command: set print source-on-step false

Default setting: true
Arguments: true or false

Description:
Determines if the source line of the application is printed on step/next.

set break pending

Summary: Set behavior of setting pending breakpoints.

Default setting: auto
Arguments: auto or on or off

Description:
Set if pending breakpoints are enabled. If enabled an unrecognized breakpoint location will create a pending breakpoint. If disabled an unrecognized breakpoint location results in an error. If auto an unrecognized breakpoint location results in a user query to see if a pending breakpoint should be created.

maint set earlyentry

Summary: Set early entry mode.
Usage: maint set earlyentry [on|off]
Example command: maint set earlyentry on

Description:
Set debugger to bypass continuing to application code on launch/attach. The application will be left in the PMI_Init() function upon startup.

maint set unsafe

Summary: Set unsafe execution mode.
Usage: maint set unsafe [on|off]
Example command: maint set unsafe on

Description:
Set debugger to bypass certain internal checks. This mode should only be modified by maintainers.

maint sync

Summary: Wait for an async response from each rank.
Usage: maint sync\

Description:
If synchronous event mode is set (maint set sync on), wait for an async response from all ranks. No-op if maint set sync is off.

maint set sync

Summary: Set synchronous event mode.
Usage: maint set sync [on|off]
Example command: maint set sync on

Description:
Set debugger to treat asynchronous events as synchronous. This will cause commands such as continue to block input until all ranks affected by the command to report an asynchronous event. Use with caution! If a rank is not expected to reach a breakpoint, a deadlock can occur.

show cuda

Summary: show cuda specific settings.
Usage: show cuda [subcommand]
Example: show cuda memcheck
Description:
Show cuda specific settings. This will only be available when a cuda application is launched or attached with the –gpu flag. Use with no arguments for the available subcommands.

show directories

Summary: Display source search path.

Description:
Display the current search path for locating source files. A $cwd in the path means the current working directory. A $cdir in the path means the compilation directory of the source file.

show error

Summary: Warn and error epsilon for floating point comparison.

Description:
Display the current tolerance level setting for compare/assert command difference deltas when comparing floating point values.

show function

Summary: Function definition information.

Description:
Display a list of the currently defined function macros.

show environment

Summary: Environment variable information.

Description:
Display the environment variables that were set by calling the set environment command.

show version

Summary: Version information.

Description:
Display the debugger version number.

show prompt

Display the debugger prompt.

show scheduler locking

Summary: Show the behavior of the thread schedule locking during execution
Usage: show scheduler-locking
Example command: show scheduler-locking

Description:
Show the mode for the locking scheduler:
off : no locking, threads may preemt any time
on : full locking, no thread except the current thread may run
step : scheduler is locked during stepping commands (step,next,stepi,nexti)
replay : scheduler is locked during replay mode and unlocked during normal execution.

show non stop

Summary: Show the behavior of the thread schedule locking during execution
Usage: show non-stop
Example command: show non-stop

Description:
Show the non-stop mode for thread stopping:
off : stopping one thread in a process stops all threads
on : other threads continue running when one is stopped

show subcmd args help

Summary: Show arguments that will be passed to new processes upon a “run” command.
Usage: show args
Example command: show args

Description:
Shows arguments that will be passed to new launches upon a “run” command. By default, stored arguments are set to those used upon initial job launch.

show print entryframe

Summary: Printing of frame on entry mode information.

Description:
Display if the frames of the application ranks are printed on launch/attach.

show print sourceonstep

Summary: Printing of source on step information.

Description:
Display if the source line of the application is printed on
step/next.

show break pending

Summary: Show behavior of setting pending breakpoints.

Description:
Display if pending breakpoints are enabled. If enabled an unrecognized breakpoint location will create a pending breakpoint. If disabled an unrecognized breakpoint location results in an error. If auto an unrecognized breakpoint location results in a user query to see if a pending breakpoint should be created.

maint show earlyentry

Summary: Early entry mode information.

Description:
Display the current setting of early entry mode.

maint show unsafe

Summary: Unsafe mode information.

Description:
Display the current setting of unsafe mode.

maint show sync

Summary: Synchronous event mode information.

Description:
Display the current setting of synchronous event mode.

unset environment

Summary: Unset environment variable value.
Usage: unset environment <name>
Example command: unset environment var

Arguments: name (environment variable name)

Description:
Unset the environment variable named name from the environment. This can only be used to unset environment variables that were set using the set environment command. This will not affect any previously launched applications and will only take effect upon the next launch command.

show print elements

Summary: Number of elements to print when an array or c-string is printed.

Description:
Display the number of elements to print when an array or c-string is printed.

set print elements

Summary: Set the amount of elements to be printed in arrays or c-strings.
Usage: set print elements [<n>|”unlimited”]
Example command: set print elements 200

Default setting: 0 (unlimited)
Arguments: a positive integer or “unlimited”.

Description:
Sets the amount of elements to be printed in arrays or c-strings. Setting to “unlimited” will print unlimited elements (until a null terminator or similar is found)

show print pretty

Show if indented structure printing is enabled.

set print pretty

Summary: Enable or disable indented printing for structures.
Usage: set print pretty [“on”|”off”]
Example command: set print pretty on

Default setting: off

Description:
When pretty printing is enabled, structures and classes will be printed on multiple lines with indenting.

set print cache last

Summary: Enable or disable value caching when using the print command.
Usage: set print value-caching [“on”|”off”]
show print value-caching

Default setting: off

Description:
When value caching is enabled, every value printed with the print command will be cached in the debug variable “$”. You can then use “$” like any other value.

Example:
dbg all&gt; print one_d_i
app{0..2}: {1,2,3,4,5,6,7,8,9,10}
dbg all&gt; print $
app{0..2}: {1,2,3,4,5,6,7,8,9,10}
dbg all&gt; print intarray + $
app{0..2}: {2,4,6,8,10,12,14,16,18,20}
dbg all&gt; print $
app{0..2}: {2,4,6,8,10,12,14,16,18,20}

Enabling value caching has a performance cost and is disabled by default.

usage decomposition

Summary: Decomposition commands.

Description:
Display help information regarding the use of decomposition commands. The decomposition commands are only valid while defining a decomposition in script mode.

blockmap dimension

Summary: Decomposition dimension command.
Available only when building a decomposition.
Usage: dimension <dimensional definition>
Example command: dimension 32,64,1024

Arguments: dimensional definition (rank/size information)

Description:
The dimension definition is used to describe the dimensionality (or rank) and size of the global decomposition.

The dimension command accepts a comma-separated numerical list to define the decompositions’s global data dimension and size in each dimension. For example, a dimension definition of 32,64,1024 would define a three dimensional global array of size 32x64x1024. The first entry in the list represents the first dimension of the global reconstruction, the second entry represents the second dimension, and so on.

The dimensionality of each provided decomposition characteristic must match. If a dimension definition of 32,64,1024 is supplied, then three dimensional distribute, proc_grid, and dim_order definitions must also be provided to describe the characteristics of each of the three dimensions.

blockmap distribute

Summary: Decomposition distribute command.
Available only when building a decomposition.
Usage: distribute <dimensional definition>
Example command: distribute *,block,block

Arguments: dimensional definition (distribution type information)

Description:
The distribute definition is used to describe the distribution scheme for each dimension of the global decomposition.

The distribute definition accepts a comma-separated list of distribution types to define each dimension of the global decomposition. Valid distribution types are block, cyclic, a numerical value, or an asterisk *.

The block keyword is used when equal sized contiguous chunks of data are assigned to each rank. The size of the chunk is determined by dividing the dimension by the proc_grid size for the dimension.

The cyclic keyword is used when elements in the dimension are dealt out in round robin fashion. The number of ranks that are dealt over is determined by the proc_grid size for that dimension.

A numerical value is used to describe a block-cyclic distribution scheme. The numerical value represents the blocking factor used to partition the global decomposition’s dimension into contiguous blocks onto which the process grid is mapped.

The * value indicates that the distribution scheme for that dimension is ignored. Each rank in the global decomposition will contain all of the data in that dimension. This can be thought of as columns (or rows) of data for that particular dimension.

For * dimensions, the * keyword should also be provided to the proc_grid and dim_order definitions for that dimension because there is no distribution present. No ranks are assigned to that particular dimension (every rank contains that dimension’s data).

The dimensionality of each provided decomposition characteristic must match. If a distribute definition of *,block,block is supplied, then three dimensional dimension, proc_grid, and dim_order definitions must also be provided to describe the characteristics of each of the three dimensions.

blockmap proc grid

Summary: Decomposition process grid command.
Usage: proc_grid <dimensional definition>
Example command: proc_grid *,4,64

Arguments: dimensional definition (process rank information)

Description:
The process grid definition is used to describe the process grid (or number of ranks contained in each dimension) for the decomposition.

The process grid definition accepts a comma-separated list of numerical values or asterisks * to define the number of ranks distributed in each dimension of the global decomposition.

For example, a process grid definition of *,4,64 would define a process grid of 256 ranks (1 x 4 x 64=215) with no ranks distributed in the first dimension, four ranks distributed in the second dimension, and sixty four ranks distributed in the third dimension. As with the dimension definition, the first entry in the list represents the first dimension in the process grid of the global decomposition and so on.

The * value represents that the distribution scheme for that dimension is ignored. Each rank in the global decomposition will contain all of the data in that dimension. This can be thought of as columns (or rows) of data for that particular dimension.

For * dimensions, the * keyword should also be provided to the distribute and dim_order definitions for that dimension because there is no distribution present. No ranks are assigned to that particular dimension (every rank contains that dimension’s data).

The dimensionality of each provided decomposition characteristic must match. If a proc_grid definition of *,4,64 is supplied, then three dimensional dimension, distribute, and dim_order definitions must also be provided to describe the characteristics of each of the three dimensions.

blockmap dim order

Summary: Decomposition dimension order command.
Available only when building a decomposition.
Usage: dim_order <dimensional definition>
Example command: dim_order *,1,2

Arguments: dimensional definition (rank assignment information)

Description:
The dimension order definition is used to describe the fastest varying dimensions in the global decomposition for rank assignment (i.e., which dimension contains sequential numbered ranks; and the order in which the other dimensions are incremented).

The dimension order definition accepts a comma-separated list of numerical values or asterisks * to define the order in which application ranks are assigned in each dimension of the process grid for the global decomposition.

Each local “chunk” of data obtained from each rank needs to be placed into the global decomposition. In order to do this, each rank needs to be assigned a logical position in the process grid for its “chunk” of data. When dealing with n-dimensional distributions, any of the n dimensions could be assigned sequential numbered ranks and any of the other higher order dimensions could be incremented after the dimension containing contiguous ranks is filled.

To define this order, each of the distributed dimensions is assigned a sequential number, starting from 1 up to n, to describe the fastest-to-slowest varying dimensions.

The fastest varying dimension is the dimension that is assigned contiguous ranks up to its corresponding process grid size. The second fastest varying dimension is incremented after the fastest varying dimension is completely filled and ranks are again assigned to the fastest varying dimension. This process continues until all ranks have been assigned to all of the n-dimensions.

Any dimension that is not distributed should receive a * for its corresponding dimension order dimension because all of the data in that dimension is present in every rank. For * dimensions, the * keyword should also be provided to the distribute and proc_grid definitions for that dimension because there is no distribution present.

If there is only one distributed dimension, then the dim_order definition is optional because there is only a single dimension and assignment order is constant.

The dimensionality of each provided decomposition characteristic must match. If a dim_order definition of *,1,2 is supplied, then three dimensional dimension, distribute, and proc_grid definitions must also be provided to describe the characteristics of each of the three dimensions.

blockmap end

Summary: Decomposition end command.
Usage: end
Example command: end

Arguments: None.

Description:
This command is used to end the current decomposition definition. After issuing end, the debugger will exit script mode and attempt to create a decomposition function for future use. The user will be warned if there were any errors that prevented the decomposition function from being created.

script set

Summary: Set assertion script options”
Usage: set
: set <option> <value> [<value>, <value>…]”
Description:
Set various assertion script options while writing an assertion script. If used with no arguments, a table of all available settings is printed with their current values. Each setting has a scope: “per assert” or “per script”. “per assert” settings will apply to each assert written after the set command, and none before. “per script” settings will apply to the entire script, even to asserts written before the set command was given.

script show

Summary: Show current values of assertion script options”
Usage: show
: show <option>”
Description:
Show the current values of various assertion script options. If used with no arguments, a table of all available settings is printed with their current values. Each setting has a scope: “per assert” or “per script”. “per assert” settings will apply to each assert written after the set command, and none before. “per script” settings will apply to the entire script, even to asserts written before the set command was given.

save script

Summary: Save assertion script state to a file.
Usage: save scripts <filename>
save script $name <filename>

Saves assertion script(s) to a file on disk. Use the source command to restore them.

The saved file contains plaintext gdb4hpc commands and can be edited with any
text editor.

script assert

Summary: Assertion script assert command.
Available only when building a script.
Usage: assert <expr1> = <expr2>
Example command:
assert $app1::var1@”app1.c”:10 = $app2::var2@”app2.c”:20

Arguments: expr1 (assert expr syntax), expr2 (assert expr syntax)

Assert expression syntax

Expr Usage: <procset>::<var>@”<source file>”:<line>
Expr arguments: procset (defined process set name), var (application
variable), source file (quoted source file name), line (line in source file to reference var)
Expr example: $my_app::my_var@”my_source.c”:10

Description:
Assert that the variable var defined in expr1 is equivalent to the variable var defined in expr2. Variables are specified using the assert expression syntax, which declares both spatial and temporal references for a variable.

Define the spatial locality with $procset::var to specify the particular variable of interest within a set of application ranks. This is followed with the temporal reference of “source file”:line to define the point within application execution where to examine the value of the variable.

The result of the assertion command depends on the current settings of error, display, output, and logging.

See help build

script end

Summary: Assertion script end command.
Usage: end
Example command: end

Arguments: None.

Description:
This command is used to end construction of the current assertion script. After issuing end, the debugger will exit script mode and attempt to compile the assertion script. The user will be warned if there were any errors that prevented the assertion script from being compiled. Upon successful compilation, the assertion script can be run by issuing the start command.

set command timeout

Summary: Set timeout for blocking commands.
Usage: set command-timeout <seconds>
Example command: set command-timeout 30

Argument: Duration in seconds.

Description:
Set an automatic time out for blocking commands. When set to zero or a negative number, no timeout is applied, and blocking commands will wait forever. This timeout also applies to the quit command.

show command timeout

Summary: Show timeout for blocking commands.
Usage: show command-timeout

Arguments: None.

Description:
Show the current amount of time that a blocking command will be cancelled after.

set height

Summary: Set paging threshold
Usage: set height <n>
Example commands: set height 40
set height 0

Description:
Set the threshold for activating the external pager (see set pager command). If certain commands (e.g. backtrace) have a result with more lines than the currently set height, gdb4hpc will use the currently set pager to view the result instead of the gdb4hpc terminal.

gdb4hpc will automatically detect the terminal height on startup and set this setting to match the terminal. Paging will still be disabled until the set pager command is used, and paging is always disabled in batch mode.

Set to 0 to disable paging.

show height

Summary: Show paging threshold
Usage: show height

Show the threshold for activating the external pager (see set pager
command). If certain commands (e.g. backtrace) have a result with more
lines than the currently set height, gdb4hpc will use the currently set
pager to view the result instead of the gdb4hpc terminal.