What’s New in gdb4hpc 4.16.4

Here’s an overview of the new features and improvements in gdb4hpc 4.16.4. For more details about each feature, see the full gdb4hpc documentation.

CUDA Debugging

gdb4hpc 4.16.4 features new improvements to the CUDA debugging experience.

The following cuda command has the same syntax as the CUDA settings in cuda-gdb. See the cuda-gdb documentation for details on what’s available: https://docs.nvidia.com/cuda/cuda-gdb/index.html.

CUDA API Error Reporting

gdb4hpc can now detect and report CUDA API errors:

dbg all> continue
a{0..1}: warning: Cuda API error detected: cudaMalloc returned (0x1)

gdb4hpc can also be configured to stop the application if a CUDA API error is detected:

dbg all> set cuda api_failures stop
dbg all> continue
a{0..1}: Cuda API error detected: cudaMalloc returned (0x1).
dbg all> backtrace -2
a{0..1}: #0 main at ./src/cuda/simpleMPI_error.cpp:99
a{0..1}: #1 computeGPU at ./src/cuda/simpleMPI_error.cu:65

Use the set cuda api_failures stop command to enable this feature. Other options are also available, see https://docs.nvidia.com/cuda/cuda-gdb/index.html#checking-api-errors.

Break on CUDA Kernel Launch

gdb4hpc can now stop the application when a CUDA kernel is launched:

dbg all> cuda set break_on_launch application
dbg all> continue
dbg all> <$a>: Running on 2 nodes
a{0..1}: In simpleMPIKernel at /home/users/adangelo/gdb4hpc-tests/src/cuda/simpleMPI.cu:55

Use the set cuda break_on_launch command to enable this feature. See https://docs.nvidia.com/cuda/cuda-gdb/index.html#set-cuda-break-on-launch.

Python Debugging

gdb4hpc 4.16.4 features new improvements to the Python debugging experience.

Additional Python Versions Supported

gdb4hpc now supports debugging Python 3.5, 3.7, and 3.13.

Breakpoint Verification

gdb4hpc will now verify that the location of a specified breakpoint in Python mode actually exists before setting it. If it doesn’t exist, it will be treated like a pending breakpoint.

python all> break 10
a{0}: Debugger error: Line 10 is past end of current python source file.
Make breakpoint pending on future code object load? ([y] or n) y
a{0}: Breakpoint 8: Pending at lib.py:10.

Breakpoints in Multithreaded Applications

gdb4hpc now supports setting breakpoints in any thread in a multithreaded Python application. Previously, gdb4hpc only supported setting breakpoints in the main thread.

Frame Command and Up/Down Command

gdb4hpc now supports the usual frame, up, and down commands in Python mode.

python all> frame
a{0}: #0  sleep at /src/python/scopes.py:9
python all> up
a{0}: #1  main at /src/python/scopes.py:14
python all> p pi
a{0}: 3.14159
python all> up
a{0}: #2  <module> at /src/python/scopes.py:19
python all> p e
a{0}: 2.718

List Command

gdb4hpc now supports the usual list command in Python mode.

python all> list
a{0}: 4    	import sys
a{0}: 5
a{0}: 6    	globl = "a grumpy global string"
a{0}: 7
a{0}: 8    	def sleep(seconds):
a{0}: 9    	    local = "a friendly neighborhood string"
a{0}: 10   	    time.sleep(seconds)
a{0}: 11
a{0}: 12   	def main():
a{0}: 13   	    pi = 3.14159
a{0}: 14   	    sleep(int(sys.argv[1]))

Exception Support

gdb4hpc will now print a backtrace before a Python application exits due to an uncaught exception:

python all> continue
a{0}: Uncaught python exception:
KeyError: 'oofdah'
  <module> at /src/python/exceptions.py:9
  main at /src/python/exceptions.py:6
  oofdah at /src/python/exceptions.py:3
a{0}: Python will now exit.

gdb4hpc can also stop the application before any Python exception is thrown by using the catch raise command:

python all> catch raise
a{0}: Catchpoint 2: Set on raise.
python all> continue
a{0}: Application raised an exception:
KeyError: 'oofdah'
  erroneous_fn at src/python/exceptions.py:12
python all> backtrace
a{0}: #2  <module> at src/python/exceptions.py:17
a{0}: #1  main at src/python/exceptions.py:6
a{0}: #0  erroneous_fn at src/python/exceptions.py:12

C/C++/Fortran Debugging

Printing NaN and Infinity Values

When debugging and application that uses floating point numbers, gdb4hpc now prints nan and infinity values correctly:

dbg all> print var
a{0..1}: inf
dbg all> print var_array
b{0..1}: {inf,-inf,-nan,-inf,-nan}
dbg all> compare $b{0}::var == $b{1}::var
{0}: Comparison is true.
dbg all> set $b{1}::var_array[1] = 1.0
b{1}: 1
dbg all> compare $b{0}::var_array == $b{1}::var_array
{0}: Comparison is false.
The difference delta is:
{0}: [1]:inf

Support Paging the info threads Command

GPU applications can have thousands of threads. gdb4hpc now supports setting up a pager via the set pager command that will nicely contain the result of the info threads command if there are too many threads.

The set pager command also affects other commands that can have long results, like backtrace or print.

Better Handling of Unregistered Breakpoints and Watchpoints

Previously, gdb4hpc didn’t handle breakpoints set in gdbmode properly:

# Old behavior
dbg all> gdb break do_inc
a{0}: Breakpoint 28 at 0x201a2b: file src/c/c_inc.c, line 5.
dbg all> continue
a{0}: Debugger error: Hit unknown breakpoint!

gdb4hpc handles these correctly now:

# New behavior
dbg all> gdb break do_inc
a{0}: Breakpoint 28 at 0x201a2b: file src/c/c_inc.c, line 5.
dbg all> continue
a{0}: In do_inc at /src/c/c_inc.c:5
5: backtrace
a{0}: #1  main at /src/c/c_inc.c:13
a{0}: #0  do_inc at /src/c/c_inc.c:5

The gdb4hpc Visual Studio Code Extension

gdb4hpc 4.16.4 features these improvements to the gdb4hpc Visual Studio Code Extension (https://marketplace.visualstudio.com/items?itemName=HPE-CDST.gdb4hpc):

  • The step out button has been implemented

  • The pause button has been implemented

  • Commands can be forwarded directly to gdb4hpc by entering them in the debug console window

  • Breakpoints can now be set via the gutter in the source code window

  • Shell commands can now be run before Visual Studio Code launches gdb4hpc. These can be used to set up the environment that gdb4hpc runs in (e.g. loading environment modules)

Bug Fixes and Improvements

In addition to the features listed here, there are also plenty of bug fixes and under-the-hood improvements to gdb4hpc in the 4.16.4 release:

  • Fix cuda command failures in some cases

  • Fix failure to launch application when using certain acceleration runtimes

  • No not generate missing MPI breakpoint if a program uses mpi_init_thread

  • Use main function from debug information for entry breakpoint

  • Fix printing std::unique_ptr with newer stdc++

  • Fix print of character arrays shorted by set print elements

  • Fix backend gdb crash on vector<vector<string>>[][][]

  • Fix ccdb crash on startup