No functional change, all these are in comments.

Insure that output appears in a fixed order for parsing by test.

This supports heterogeneous environments (i.e. different
MPS counts on different GPUs within a node)

if the gres count on a node with topology changes when the slurmctld
restarts then the gres data structures were left in an inconsistent
state. Namely the bitmaps would reflect the old size while the count
reflects the new size, which resulted in asserts. In addition, the
gres/mps data structure sizes need to match the gpu count on each
node. This new logic will synchronize mps data structures on gpu count
changes.

Add support for co-scheduling of gres/gpu and gres/mps.
  GPUs that are allocated to one are avoided for the other
  GRES type.
Add gres/mps documentation
Recover job gres/mps state on slurmctld restart. Wwe need to use
  job gres/mps state to recover node info since we will not know
   the count of mps on each device file until the node registers

Check for cgroup usage and change GPU indexes accordingly.
Fix formatting errors in docs.
bug 5520

Update slurm.spec and slurm.spec-legacy as well.

Bug 6029.

Duplicate file names will cause problems for gres/mps,
which needs to make 1-to-1 to gres/gpu devices

Support undocumented "Files=" in addition to "File=".
Note that multiple file name can be used as an argument
and this minor change eliminates some possible confusion.

bug 5520

bug 5520

without this, the jobs were being assigned the wrong
CUDA_VISIBLE_DEVICES value

The cpu frequency set by the user is not exact with current kernels.
There seems to be a fair variation depending upon timing and other
events. This is resulting in test1.76 failing sporatically. This
changes the logic to retry if the frequency differs by more than
10 percent rather than failing immediately.

The device numbers are set using the same mechanism used to set
CUDA_VISIBLE_DEVICES

bug 5520

Add step_unconfigure_hardware() to GRES plugin API
Update test39.18 regarding links.
Update GRES docs.
Update docs related to links.
Document GPU frequency resetting behavior.
Specify what the default is for GpuFreqDef.
Move NVML init and shutdown to configure() and unconfigure().
Get rid of superfluous `!= 0`-style statements.
Print note when GPU index != minor number.
Clean up various formatting and other errors.

bug 5520

No functional change.

Due to upcoming changes in the X11 forwarding subsystem, support
for older-style X11 tunnels will be removed. Older client commands
cannot support the newer style. Rather than have the tunnel fail,
request the job allocation request up front.

Bug 3647.

Also tweak the one info() message here to match these others.

New X11 forwarding code will only support forwarding back to
salloc or an allocating srun command.

Using this option within sbatch was always hit-or-miss. If the
user submitting was disconnected from the alloc host for any
reason their xauth credentials would likely fail even if they
managed to get assigned the same local TCP port for forwarding.

Bug 3647.

Bug 6029

< PAM_MAX_MSG_SIZE (which as of this date is 512)