# Debug Options

([Main README](README.md))

There are many compiler flags and environment variables that can be useful for debugging GPU applications. You will find a couple down below.

## Flags

**-hmsgs** and **-hlist=m**  
Indicate what regions are offloaded.

## Environment variables

**CRAY_ACC_DEBUG**  
Emits messges for all offloading operations

- Value 1: Concise info on offloading regions
- Value 2: More in-depth runtime info, but user friendly.
- Value 3: Very verbose (i.e. memory addresses), not designed for everyday user.

**LIBOMPTARGET_INFO**  
Controls what's printed, but not how much

- Value is a bitfield: https://openmp.llvm.org/design/Runtimes.html#libomptarget-info
- Set to -1 to get it all.

**LIBOMPTARGET_DEBUG**  
Like LIBOMPTARGET_INFO, but much more info (probably too much)

**AMD_SERIALIZE_KERNEL**  
Good for checking kernel race conditions.

- Value 1: Synchronize before launces (i.e. make sure everything is done on GPU)
- Value 2: Synchronize after launces (i.e. wait for kernel to finish before moving one)
- Value 3: Do both 1 and 2

**AMD_SERIALIZE_COPY**  
Good for checking kernel race conditions.

- Value 1: Synchronize before copies (i.e. make sure everything is done on GPU)
- Value 2: Synchronize after copies (i.e. wait for kernel to finish before moving one)
- Value 3: Do both 1 and 2

**MPICH_ENV_DISPLAY=1**  
Logs exactly how MPI was configured

**MPICH_VERSION_DISPLAY=1**  
Helps to track how issues change with different MPI versions

### DO NOT USE IN PRODUCTION

**MPICH_SINGLE_HOST_ENABLED=0**  
Force traffic through the NIC when you only have one node

**MPICH_SMP_SINGLE_COPY_MODE=NONE**  
Disables all intranode optimizations and forces all in-node communication through slow interprocess communicator paths on CPU memory.

**MPICH_GPU_IPC_ENABLED=0**  
Disabling SMP operations, also disables IPC.
