netCDF - Network Common Data Form

For detailed documentation, refer to the upstream resources:

Overview

NetCDF (network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. NetCDF data is self-describing (files include metadata about the data they contain) and portable (files can be read on any platform regardless of how integers, characters, and floating-point numbers are stored).

SUPS provides:

Package	Version	Install Prefix
netCDF-C	4.10.0	`/opt/cray/sups/netcdf-c/4.10.0`
netCDF-CXX4	4.3.1	`/opt/cray/sups/netcdf-cxx4/4.3.1`
netCDF-Fortran	4.6.2	`/opt/cray/sups/netcdf-fortran/4.6.2/<compiler>`
netCDF-C (parallel)	4.10.0	`/opt/cray/sups/netcdf-c-parallel/4.10.0`
netCDF-CXX4 (parallel)	4.3.1	`/opt/cray/sups/netcdf-cxx4-parallel/4.3.1`
netCDF-Fortran (parallel)	4.6.2	`/opt/cray/sups/netcdf-fortran-parallel/4.6.2/<compiler>`

Where <compiler> is one of amd, aocc, nvidia, cray, or gnu. The netCDF-C and netCDF-CXX4 libraries are compiler-independent. The netCDF-Fortran libraries are compiler-specific because Fortran module files are not portable across compilers.

The serial libraries are compiled with the stack-native compilers (ROCm LLVM compilers amdclang, amdclang++, amdflang or NVHPC compilers nvc, nvc++, nvfortran) and linked against HDF5 2.1.0 (serial). The parallel libraries are compiled with Cray MPICH wrappers and linked against HDF5 2.1.0 (parallel), enabling MPI-based parallel I/O. DAP remote data access is disabled. The libraries are installed inside the container only.

Shell variables used in this document

All examples in this document use the AMD Rocky image and AMD ROCm LLVM compilers. Substitute sups-nvidia-rocky (or the appropriate SLES variant) and the NVHPC compiler paths (nvc, nvc++, nvfortran under the NVHPC install prefix) when working with the NVIDIA stack.

Set these once in your shell before running any of the commands below:

export CONTAINER_NAME="sups-amd-rocky"    # image name -- substitute your vendor/distro
export SUPS_DIR="/path/to/sups"           # path to the un-tarred SUPS distribution directory
export WORKDIR="/home/username/myproject" # path to your scripts and data
eval "$(sups-config --get SUPS_IMAGE_DIR)"  # set SUPS_IMAGE_DIR
eval "$(sups-config --get SUPS_COMMON_DIR)" # set SUPS_COMMON_DIR
export DEV_CONTAINER="${SUPS_IMAGE_DIR}/singularity/${CONTAINER_NAME}.sif"
export LOCAL_SCRATCH="/tmp/${USER}"      # node-local scratch; must not be on a network filesystem
export DEV_SANDBOX="${LOCAL_SCRATCH}/netcdf-sandbox" # writable sandbox for manual installs

Create the scratch directory and confirm it is genuinely node-local before building the sandbox:

mkdir -p "${LOCAL_SCRATCH}"
findmnt -T "${LOCAL_SCRATCH}"   # FSTYPE must not be nfs*, gpfs, lustre or panfs

See Writable sandboxes for why this matters, including the diskless-node case where /tmp is a RAM-backed tmpfs.

SUPS_DIR points to the directory produced by un-tarring the SUPS distribution tarball (e.g. the directory that contains tools/hpc-container-wrapper). It is mounted read-only at /opt/sups inside the container.

SUPS_COMMON_DIR points to the directory containing the locally built RPM repositories (e.g. sups-oss). It is mounted read-only at /var/sups/common inside the container so that dnf/zypper can find the SUPS packages.

WORKDIR points to your own working directory (scripts, input data, output files). It is mounted at /workspace inside the container.

Install netCDF

The netcdf-c and netcdf-fortran RPMs are provided by the sups-oss repository, which is built locally as part of the SUPS build process. They are not available from any upstream RPM source. The SUPS reference container image includes both packages by default.

If you need to install netCDF manually in a development container that has the sups-oss repository configured:

apptainer build --fakeroot --sandbox ${DEV_SANDBOX} ${DEV_CONTAINER}
apptainer shell --writable --fakeroot \
  --mount type=bind,src=${SUPS_COMMON_DIR},dst=/var/sups/common,ro \
  ${DEV_SANDBOX}

# For Rocky Linux 9.6 development container
dnf install netcdf-c
dnf install netcdf-cxx4
dnf install netcdf-fortran-amd

# For SLES 15 SP7 development container
zypper install netcdf-c
zypper install netcdf-cxx4
zypper install netcdf-fortran-amd

Installing packages needs a persistent writable container: build a writable sandbox from the SIF as described in Writable sandboxes. Copy the rebuilt SIF to shared storage for multi-node runs.

Installing netcdf-fortran-amd also pulls in netcdf-c and hdf5-amd as dependencies. Installing netcdf-cxx4 also pulls in netcdf-c and hdf5-amd. Substitute aocc, nvidia, cray, or gnu for the appropriate compiler toolchain.

To install the parallel (MPI-enabled) variants:

# For Rocky Linux 9.6 development container
dnf install netcdf-c-parallel
dnf install netcdf-cxx4-parallel
dnf install netcdf-fortran-parallel-amd

# For SLES 15 SP7 development container
zypper install netcdf-c-parallel
zypper install netcdf-cxx4-parallel
zypper install netcdf-fortran-parallel-amd

Once you have completed all of the installation and configuration steps in this section, exit the sandbox shell and build a new SIF from it. Use that image for the verification and run commands below, which would otherwise open the original unmodified image. The rebuild must be the last customization step — anything done afterwards will not be in the SIF. Build it into WORKDIR rather than the node-local scratch directory, so that the compute nodes in the multi-node examples below can read it:

apptainer build --fakeroot "${WORKDIR}/netcdf-container.sif" "${DEV_SANDBOX}"
export DEV_CONTAINER="${WORKDIR}/netcdf-container.sif"

The parallel packages pull in hdf5-parallel-amd (and netcdf-c-parallel where applicable) as RPM dependencies. They are built against Cray MPICH, so a Cray MPICH installation must be present at runtime to compile and run parallel programs. Substitute the appropriate compiler suffix (aocc, nvidia, cray, or gnu) for other toolchains.

Verifying the Installation

Use nc-config and nf-config to confirm the libraries are installed and display their build configuration:

apptainer shell ${DEV_CONTAINER}

# netCDF-C version and build info
/opt/cray/sups/netcdf-c/4.10.0/bin/nc-config --version
/opt/cray/sups/netcdf-c/4.10.0/bin/nc-config --all

# netCDF-CXX4 version and build info
/opt/cray/sups/netcdf-cxx4/4.3.1/bin/ncxx4-config --version
/opt/cray/sups/netcdf-cxx4/4.3.1/bin/ncxx4-config --all

# netCDF-Fortran version and build info
/opt/cray/sups/netcdf-fortran/4.6.2/amd/bin/nf-config --version
/opt/cray/sups/netcdf-fortran/4.6.2/amd/bin/nf-config --all

You can also verify that the netCDF utilities are functional:

# Show the format of any netCDF file
/opt/cray/sups/netcdf-c/4.10.0/bin/ncdump -k /path/to/file.nc

Compiling and Linking C Programs

Use the ROCm LLVM C compiler with the netCDF-C installation paths. The nc-config utility provides the correct compiler and linker flags.

apptainer shell \
  --mount type=bind,src=${WORKDIR},dst=/workspace \
  ${DEV_CONTAINER}

NC_PREFIX=/opt/cray/sups/netcdf-c/4.10.0
HDF5_PREFIX=/opt/cray/sups/hdf5/2.1.0/amd

/opt/rocm/core-7.13/bin/amdclang \
    $(${NC_PREFIX}/bin/nc-config --cflags) \
    -o my_program my_program.c \
    $(${NC_PREFIX}/bin/nc-config --libs) \
    -L${HDF5_PREFIX}/lib -Wl,-rpath,${HDF5_PREFIX}/lib

Or specify the flags manually:

/opt/rocm/core-7.13/bin/amdclang \
    -I${NC_PREFIX}/include \
    -o my_program my_program.c \
    -L${NC_PREFIX}/lib -lnetcdf \
    -L${HDF5_PREFIX}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NC_PREFIX}/lib \
    -Wl,-rpath,${HDF5_PREFIX}/lib \
    -lz -lm

Compiling and Linking C++ Programs

Use the ROCm LLVM C++ compiler with the netCDF-CXX4 installation paths. The ncxx4-config utility provides the correct compiler and linker flags.

apptainer shell \
  --mount type=bind,src=${WORKDIR},dst=/workspace \
  ${DEV_CONTAINER}

NCXX4_PREFIX=/opt/cray/sups/netcdf-cxx4/4.3.1
NC_PREFIX=/opt/cray/sups/netcdf-c/4.10.0
HDF5_PREFIX=/opt/cray/sups/hdf5/2.1.0/amd

/opt/rocm/core-7.13/bin/amdclang++ \
    $(${NCXX4_PREFIX}/bin/ncxx4-config --cflags) \
    -I${NC_PREFIX}/include \
    -o my_program my_program.cpp \
    $(${NCXX4_PREFIX}/bin/ncxx4-config --libs) \
    -L${NC_PREFIX}/lib -lnetcdf \
    -L${HDF5_PREFIX}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NCXX4_PREFIX}/lib \
    -Wl,-rpath,${NC_PREFIX}/lib \
    -Wl,-rpath,${HDF5_PREFIX}/lib

Or specify the flags manually:

/opt/rocm/core-7.13/bin/amdclang++ \
    -I${NCXX4_PREFIX}/include \
    -I${NC_PREFIX}/include \
    -o my_program my_program.cpp \
    -L${NCXX4_PREFIX}/lib -lnetcdf-cxx4 \
    -L${NC_PREFIX}/lib -lnetcdf \
    -L${HDF5_PREFIX}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NCXX4_PREFIX}/lib \
    -Wl,-rpath,${NC_PREFIX}/lib \
    -Wl,-rpath,${HDF5_PREFIX}/lib \
    -lz -lm

Example: Create and Read a NetCDF-4 File (C++)

The following program creates a netCDF-4 file with a 2D variable, writes data to it, then reads the data back.

#include <iostream>
#include <vector>
#include <netcdf>

using namespace netCDF;

int main() {
    const int NX = 4, NY = 6;

    // Create a netCDF-4 file
    NcFile dataFile("example.nc", NcFile::replace);

    // Define dimensions
    NcDim xDim = dataFile.addDim("x", NX);
    NcDim yDim = dataFile.addDim("y", NY);

    // Define a 2D variable of type float
    NcVar data = dataFile.addVar("data", ncFloat, {xDim, yDim});
    data.putAtt("units", "meters");

    // Write data
    std::vector<float> values(NX * NY);
    for (int i = 0; i < NX * NY; i++)
        values[i] = static_cast<float>(i) * 0.5f;
    data.putVar(values.data());

    // File is automatically closed when dataFile goes out of scope
    std::cout << "Wrote example.nc" << std::endl;

    // Read it back
    NcFile readFile("example.nc", NcFile::read);
    NcVar readVar = readFile.getVar("data");
    std::vector<float> readValues(NX * NY);
    readVar.getVar(readValues.data());

    std::cout << "Read back " << readValues.size() << " values, "
              << "first=" << readValues[0] << " last=" << readValues.back()
              << std::endl;

    return 0;
}

Compile and run:

NCXX4_PREFIX=/opt/cray/sups/netcdf-cxx4/4.3.1
NC_PREFIX=/opt/cray/sups/netcdf-c/4.10.0
HDF5_PREFIX=/opt/cray/sups/hdf5/2.1.0/amd

/opt/rocm/core-7.13/bin/amdclang++ \
    -I${NCXX4_PREFIX}/include \
    -I${NC_PREFIX}/include \
    -o netcdf_example netcdf_example.cpp \
    -L${NCXX4_PREFIX}/lib -lnetcdf-cxx4 \
    -L${NC_PREFIX}/lib -lnetcdf \
    -L${HDF5_PREFIX}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NCXX4_PREFIX}/lib \
    -Wl,-rpath,${NC_PREFIX}/lib \
    -Wl,-rpath,${HDF5_PREFIX}/lib \
    -lz -lm

./netcdf_example

Compiling and Linking Fortran Programs

Use the ROCm LLVM Fortran compiler with both the netCDF-Fortran and netCDF-C installation paths. The nf-config utility provides the correct flags.

apptainer shell \
  --mount type=bind,src=${WORKDIR},dst=/workspace \
  ${DEV_CONTAINER}

NF_PREFIX=/opt/cray/sups/netcdf-fortran/4.6.2/amd
NC_PREFIX=/opt/cray/sups/netcdf-c/4.10.0
HDF5_PREFIX=/opt/cray/sups/hdf5/2.1.0/amd

/opt/rocm/core-7.13/bin/amdflang \
    $(${NF_PREFIX}/bin/nf-config --fflags) \
    -o my_program my_program.f90 \
    $(${NF_PREFIX}/bin/nf-config --flibs) \
    -L${NC_PREFIX}/lib -lnetcdf \
    -L${HDF5_PREFIX}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NF_PREFIX}/lib \
    -Wl,-rpath,${NC_PREFIX}/lib \
    -Wl,-rpath,${HDF5_PREFIX}/lib \
    -lz -lm

Or specify the flags manually:

/opt/rocm/core-7.13/bin/amdflang \
    -I${NF_PREFIX}/include \
    -o my_program my_program.f90 \
    -L${NF_PREFIX}/lib -lnetcdff \
    -L${NC_PREFIX}/lib -lnetcdf \
    -L${HDF5_PREFIX}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NF_PREFIX}/lib \
    -Wl,-rpath,${NC_PREFIX}/lib \
    -Wl,-rpath,${HDF5_PREFIX}/lib \
    -lz -lm

[!IMPORTANT] The Fortran netCDF library is libnetcdff (with two f’s). The C library is libnetcdf (one f). The Fortran library depends on the C library, so both must be linked.

Compiling and Linking with Parallel netCDF (MPI)

The parallel variants of netCDF are built against Cray MPICH and HDF5-parallel, enabling collective and independent MPI-IO. Use the MPI compiler wrappers from Cray MPICH.

Set up the environment:

export MPICH_DIR=/opt/cray/pe/mpich/9.2.0/ofi/amd/7.13
export PATH="${MPICH_DIR}/bin:${PATH}"

Parallel C

NC_PAR=/opt/cray/sups/netcdf-c-parallel/4.10.0
HDF5_PAR=/opt/cray/sups/hdf5-parallel/2.1.0/amd

mpicc \
    -I${NC_PAR}/include \
    -o my_program my_program.c \
    -L${NC_PAR}/lib -lnetcdf \
    -L${HDF5_PAR}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NC_PAR}/lib \
    -Wl,-rpath,${HDF5_PAR}/lib \
    -lz -lm

Parallel C++

NCXX4_PAR=/opt/cray/sups/netcdf-cxx4-parallel/4.3.1
NC_PAR=/opt/cray/sups/netcdf-c-parallel/4.10.0
HDF5_PAR=/opt/cray/sups/hdf5-parallel/2.1.0/amd

mpicxx \
    -I${NCXX4_PAR}/include \
    -o my_program my_program.cpp \
    -L${NCXX4_PAR}/lib -lnetcdf-cxx4 \
    -L${NC_PAR}/lib -lnetcdf \
    -L${HDF5_PAR}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NCXX4_PAR}/lib \
    -Wl,-rpath,${NC_PAR}/lib \
    -Wl,-rpath,${HDF5_PAR}/lib \
    -lz -lm

Parallel Fortran

NF_PAR=/opt/cray/sups/netcdf-fortran-parallel/4.6.2/amd
NC_PAR=/opt/cray/sups/netcdf-c-parallel/4.10.0
HDF5_PAR=/opt/cray/sups/hdf5-parallel/2.1.0/amd

mpifort \
    -I${NF_PAR}/include \
    -o my_program my_program.f90 \
    -L${NF_PAR}/lib -lnetcdff \
    -L${NC_PAR}/lib -lnetcdf \
    -L${HDF5_PAR}/lib -lhdf5_hl -lhdf5 \
    -Wl,-rpath,${NF_PAR}/lib \
    -Wl,-rpath,${NC_PAR}/lib \
    -Wl,-rpath,${HDF5_PAR}/lib \
    -lz -lm

[!NOTE] The parallel netCDF libraries use nc-config, ncxx4-config, and nf-config scripts at their respective parallel prefix paths (e.g., /opt/cray/sups/netcdf-c-parallel/4.10.0/bin/nc-config). These report the correct flags for the parallel build.

Running a Parallel netCDF Program

Parallel netCDF programs must be launched with an MPI job launcher. Cray MPICH does not include mpirun or mpiexec. Use Slurm’s srun to launch MPI ranks:

srun -n 4 ./my_program

Even for single-node multi-rank runs, a Slurm allocation is required.

Using nc-config, ncxx4-config, and nf-config

The nc-config, ncxx4-config, and nf-config scripts report the compiler flags, library paths, and feature support for netCDF-C, netCDF-CXX4, and netCDF-Fortran respectively:

Command	Description
`nc-config --version`	Print the netCDF-C version
`nc-config --cflags`	Print C compiler flags (`-I...`)
`nc-config --libs`	Print linker flags (`-L... -lnetcdf`)
`nc-config --has-nc4`	Whether netCDF-4/HDF5 support is built in
`nc-config --has-dap`	Whether DAP support is built in
`nc-config --prefix`	Print the installation prefix
`ncxx4-config --version`	Print the netCDF-CXX4 version
`ncxx4-config --cflags`	Print C++ compiler flags (`-I...`)
`ncxx4-config --libs`	Print C++ linker flags (`-L... -lnetcdf-cxx4`)
`ncxx4-config --prefix`	Print the installation prefix
`nf-config --version`	Print the netCDF-Fortran version
`nf-config --fflags`	Print Fortran compiler flags (`-I...`)
`nf-config --flibs`	Print Fortran linker flags (`-L... -lnetcdff -lnetcdf`)
`nf-config --has-nc4`	Whether netCDF-4/HDF5 support is built in
`nf-config --prefix`	Print the installation prefix

NetCDF Utilities

Three command-line utilities are installed with netCDF-C under /opt/cray/sups/netcdf-c/4.10.0/bin/:

ncdump

Converts a netCDF file to human-readable CDL (Common Data Language) text.

# Show structure and coordinate values
ncdump -c data.nc

# Show only headers (dimensions, variables, attributes)
ncdump -h data.nc

# Show the file format variant
ncdump -k data.nc

# Dump specific variables
ncdump -v temperature,pressure data.nc

ncgen

Generates a binary netCDF file from a CDL text description. This is the inverse of ncdump.

# Check CDL syntax
ncgen input.cdl

# Generate a netCDF-4 file from CDL
ncgen -k nc4 -o output.nc input.cdl

# Generate C source code that creates the dataset
ncgen -l c input.cdl > create_data.c

nccopy

Copies a netCDF file, optionally changing format, compression, or chunking.

# Convert classic to netCDF-4 format
nccopy -k nc4 classic.nc netcdf4.nc

# Compress all variables (deflation level 1)
nccopy -d1 input.nc compressed.nc

# Rechunk for time-series access
nccopy -c time/1000,lat/40,lon/40 input.nc rechunked.nc

Differences from CPE

If you are transitioning from a Cray Programming Environment (CPE) system, here is how common netCDF tasks map to SUPS.

Task	CPE	SUPS
Load netCDF	`module load cray-netcdf`	Not needed - pre-installed in the container
C compiler flags	`cc` wrapper adds flags automatically	Use `nc-config --cflags` and `nc-config --libs`
Fortran compiler flags	`ftn` wrapper adds flags automatically	Use `nf-config --fflags` and `nf-config --flibs`
`nc-config` / `nf-config`	Available after `module load`	`/opt/cray/sups/netcdf-c/4.10.0/bin/nc-config` and `/opt/cray/sups/netcdf-fortran/4.6.2/<compiler>/bin/nf-config`
Parallel netCDF (MPI)	`module load cray-netcdf-hdf5parallel`	Install `netcdf-c-parallel`, `netcdf-cxx4-parallel`, `netcdf-fortran-parallel-<compiler>` - built against HDF5-parallel and Cray MPICH
`ncdump` / `ncgen` / `nccopy`	Available after `module load`	Under `/opt/cray/sups/netcdf-c/4.10.0/bin/`
DAP remote access	Enabled by default	Disabled in this build

Troubleshooting

error while loading shared libraries: libnetcdf.so

The netCDF library is not in the runtime linker search path. Either:

Add -Wl,-rpath,/opt/cray/sups/netcdf-c/4.10.0/lib when linking, or

Set LD_LIBRARY_PATH before running:

export LD_LIBRARY_PATH="/opt/cray/sups/netcdf-c/4.10.0/lib:${LD_LIBRARY_PATH:-}"

error while loading shared libraries: libnetcdf-cxx4.so

The netCDF C++ library is not in the runtime linker search path. Either:

Add -Wl,-rpath,/opt/cray/sups/netcdf-cxx4/4.3.1/lib when linking, or

Set LD_LIBRARY_PATH before running:

export LD_LIBRARY_PATH="/opt/cray/sups/netcdf-cxx4/4.3.1/lib:${LD_LIBRARY_PATH:-}"

error while loading shared libraries: libnetcdff.so

Same issue, but for the Fortran library:

export LD_LIBRARY_PATH="/opt/cray/sups/netcdf-fortran/4.6.2/amd/lib:${LD_LIBRARY_PATH:-}"

error while loading shared libraries: libhdf5.so

The HDF5 library path is also needed at runtime:

export LD_LIBRARY_PATH="/opt/cray/sups/hdf5/2.1.0/amd/lib:${LD_LIBRARY_PATH:-}"

For convenience, set all serial paths:

export LD_LIBRARY_PATH="/opt/cray/sups/netcdf-fortran/4.6.2/amd/lib:/opt/cray/sups/netcdf-cxx4/4.3.1/lib:/opt/cray/sups/netcdf-c/4.10.0/lib:/opt/cray/sups/hdf5/2.1.0/amd/lib:${LD_LIBRARY_PATH:-}"

For parallel libraries:

export LD_LIBRARY_PATH="/opt/cray/sups/netcdf-fortran-parallel/4.6.2/amd/lib:/opt/cray/sups/netcdf-cxx4-parallel/4.3.1/lib:/opt/cray/sups/netcdf-c-parallel/4.10.0/lib:/opt/cray/sups/hdf5-parallel/2.1.0/amd/lib:${LD_LIBRARY_PATH:-}"

netcdf.h: No such file or directory or netcdf.mod: No such file or directory

The compiler cannot find the netCDF headers or Fortran module files. Add the appropriate -I flag:

# For C
-I/opt/cray/sups/netcdf-c/4.10.0/include

# For Fortran
-I/opt/cray/sups/netcdf-fortran/4.6.2/amd/include

Or use nc-config --cflags / nf-config --fflags to get the correct flags automatically.

nc-config or nf-config not found

Use the full path to the script:

/opt/cray/sups/netcdf-c/4.10.0/bin/nc-config --version
/opt/cray/sups/netcdf-cxx4/4.3.1/bin/ncxx4-config --version
/opt/cray/sups/netcdf-fortran/4.6.2/amd/bin/nf-config --version

Or add the netCDF bin directories to your PATH:

export PATH="/opt/cray/sups/netcdf-c/4.10.0/bin:/opt/cray/sups/netcdf-fortran/4.6.2/amd/bin:$PATH"