The contoller will return node records with a NULL name for nodes that
are hidden. This is so that you can map a partition_info's nodes --
using it's node_inx[] -- to a node record in the returned node_array
from slurm_load_node().

Previously the perl api would leave an undefined object in the the
node_array if the hidden nodes were found before a real node in the
node_array and any hidden nodes at the end of the array from the
controller wouldn't be counted for in the perl node_array. This patch
adds empty hashes for hidden nodes and preserves the record_count and
node_array from the slurmctld.

Bug 4250

The perl api leaves undefined objects in the node_array returned by
load_nodes() for any node that is hidden.

But 4250

Bug 4003

as process exits

Bug 4003

when using xstrcasecmp. Matching up with other xstrcmp() functions.

was missing

Bug 4226

that there was no bit_fmt was out of scope on the xfree. Passing
a function address to xfree() predictably does not work very well.

Change the variable name to avoid confusion.

Bug 4241

Before:
$ sbatch --wrap="sleep 300"
Submitted batch job 228
$ squeue
             JOBID PARTITION     NAME     USER ST       TIME   CPUS NODELIST(REASON)
               228     debug     wrap    brian PD       0:00      1 (AssocMaxUnknownPerNode)

Fixed:
$ squeue
             JOBID PARTITION     NAME     USER ST       TIME   CPUS NODELIST(REASON)
               229     debug     wrap    brian PD       0:00      1 (AssocMaxCpuPerNode)

$ sacctmgr mod account stuff set maxtrespernode=cpu=-1,mem=1
$ squeue
             JOBID PARTITION     NAME     USER ST       TIME   CPUS NODELIST(REASON)
               229     debug     wrap    brian PD       0:00      1 (AssocMaxMemPerNode)

$ sbatch --wrap="sleep 300" --gres=blah:2 -pgpu
Submitted batch job 235
$ squeue
             JOBID PARTITION     NAME     USER ST       TIME   CPUS NODELIST(REASON)
               235       gpu     wrap    brian PD       0:00      1 (AssocMaxGRESPerNode)

burst_buffer/cray plugin modified to work with changes in Cray UP06
   software release.
Specific changes: Cray software now returns an error if a state_in
   or stage_out script is processed that doesn't actually request a
   stage in or out (previously silently ignored).
Also the warning message about tearing down a buffer that is already
   gone changed.

Move the check up a bit more where it'll do some good.

Bug 4184.

Bug 4146.

Bug 3467

Bug 3567

gres listed in your slurm.conf but some in gres.conf.

Bug 3974

'type' but no file defined.

plugin when constraining devices.

correctly in sacct.

A second PMI2_Init() within the same step is invalid, and cannot succeed.

Return an error code back to the client end, and close the fd to force the
step to terminate immediately.

Due to a bug in our libpmi code, just returning a cmd=response_to_init with
an appropriate rc number will not tear down the connection properly, so
send back something else that will trigger the error path.

Bug 3520.

Bug 4154.

default path.

This makes it so you don't always have to put AllowedDevicesFile in your
cgroup.conf file if your etc dir is anything other than /etc/slurm.

Adding a newline prevents this error:
conftest.c:154:8: error: if statement has empty body [-Werror,-Wempty-body]

remote cluster correctly determine the select type.

Bug 2329

If /proc was inaccessible proc_name would leak.

Put an explicit length cap in sprintf to avoid warning. The
size is checked immediate before here so this is just making
the 10-char limit explicit.

Bug 4062.

Bug 4062.

bug 3824

Do not run the Node Health Check on termination of the external
  step as this happens when the job allocation ends and the job
  NHC will be executed anyway.
Bug 4074

checked on submit.

This only mattered when submitting a job to multiple partitions.

Bug 4066

on node 0.

Bug 4035

Calling bit_unfmt() with a zero bit_size() bitmap leads to a later
call to bit_nclear() with start=0 and stop=-1, leading to the ABRT.

This scenario happened when cgroup.conf has ConstrainDevices=yes and
task_cgroup_devices_create() tries to collect the GRES devices
but gres_cpu_cnt=0, thus creating a p->cpus_bitmap = bit_alloc(gres_cpu_cnt);
of zero size which is passed by argument to bit_unfmt().

gres_cpu_cnt is 0 because we have defined a gres.conf like this:

Name=gpu Type=tesla File=/tmp/gres/tesla0 CPUs=0,1
Name=gpu Type=tesla File=/tmp/gres/tesla1 CPUs=0,1
Name=gpu Type=kepler File=/tmp/gres/kepler0 CPUs=2,3
Name=gpu Type=kepler File=/tmp/gres/kepler1 CPUs=2,3

but have no GresTypes nor GRES option in the slurm.conf / node config def.

Bug 3974

Running slurmctld under valgrind while operating with jobcomp/elasticsearch
reported the following bytes definitely lost:

==27403== 658 bytes in 1 blocks are definitely lost in loss record 301 of 342
==27403==    at 0x4C2FD4F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27403==    by 0x2281B3: slurm_xrealloc (xmalloc.c:137)
==27403==    by 0x22856A: makespace (xstring.c:114)
==27403==    by 0x2285D0: _xstrcat (xstring.c:132)
==27403==    by 0x228CE0: _xstrfmtcat (xstring.c:291)
==27403==    by 0x83C5BCD: ???
==27403==    by 0x30A913: g_slurm_jobcomp_write (slurm_jobcomp.c:172)
==27403==    by 0x18D8FC: job_completion_logger (job_mgr.c:13652)

It turns out the generated buffer in slurm_jobcomp_log_record was xstrdup'ed to
the corresponding job_node->serialized_job, but the originally generated buffer
wasn't freed afterwards. The fix consists in change the transfer so that instead
of xstrdup'ing the char * we just assign the pointer and NULL the buffer.

The job_node->serialized_job was already xfree'd properly later when the job
was indexed.

Discovered while working on Bug 4065.

Otherwise the resulting URL may be invalid. Update documentation
while here as well.

Bug 4065.

bug 4095

Given a configuration with TopologyParam including Dragonfly option, if a
job requested --switches count, the count timeout specified by either
the job request or max_switch_wait SchedulerParameters was not respected.
This was due to leaf_switch_count variable not being incremented in
_eval_nodes_dfly() function when needed, as we do in _eval_nodes_topo(),
the later being a execution path which already succeed to wait for the
switch count timeout.

Bug 4056

Coverity CID 44649

Bug 4085

instead of local.

Bug 3546