If prolog is running when attempting to signal a step, then return EAGAIN
   and retry rather than simply returning SLURM_ERROR and aborting.

Fix I/O race condition on step termination for srun launching multiple
   pack job groups. Without this change application output might be
   lost and/or the srun command might hang after some tasks exit.

Bug 3956.

Add pack_job_id and pack_job_offset to accounting database.
Modified sacct to accept pack job ID specification using "#+#" notation.
Modified sstat to accept pack job ID specification using "#+#" notation.

Bug 3999

Default to 1, unless set to 0. Allow to be set to 0 even if
GroupUpdateTime was not set before.

Move down to alphabetical position in read_config.c as well.

Bug 3956.

Fix bug in selection of GRES bound to specific CPUs where the GRES count
    is 2 or more. Previous logic could allocate CPUs not available to the job.

bug 4029

This will be fixed before 17.11, but is being left as-is on 17.02.

Bug 3956.

connection.

Bug 4009

jobcomp/elasticsearch saves/load the state to/from elasticsearch_state.  Since
the jobcomp API isn't designed with save/load state operations, the plugin
_save_state() isn't extern and not available from outside the plugin itself,
thus it is highly coupled to fini() function. This state doesn't follow the
same execution path as the rest of Slurm states, where in save_all_sate()
they are all independently scheduled. So we save it manually here on a RPC
of type REQUEST_CONTROL.

This enables that when the Primary ctld issues a REQUEST_CONTROL to the Backup
which is currently in controller mode, the Backup will save the state and when
the Primary assumes control again it can process the saved pending jobs.  The
other way around was already controlled, because when the Primary is running
in controller mode and the Backup issues a REQUEST_CONTROL, the Primary is
shutdown and when breaking the ctld main() function while(1) loop, there was
already a g_slurm_jobcomp_fini() call in place.

Bug 3908

Perform limit check on heterogeneous job as a whole at submit time to
   reject jobs that will never be able to run. Accepting pack jobs
   that can never start will have a significant effect on scheduling
   in general (blocking the queue).

When more than 1 ping cycle is spawned simultaneously (for instance
REQUEST_PING + REQUEST_NODE_REGISTRATION_STATUS for the selected nodes),
we do not track a separate ping_start time for each cycle. When ping_begin()
is called, the information about the previous ping cycle is lost. Then when
ping_end() is called for the first of the two cycles, we set ping_start=0,
which is incorrectly used to see if the last cycle ran for more than
PING_TIMEOUT seconds (100s), thus incorrectly triggering the:

 error("Node ping apparently hung, many nodes may be DOWN or configured "
       "SlurmdTimeout should be increased");

Bug 3914

 -- Add slurm.conf configuration parameters SlurmctldSyslogDebug and
    SlurmdSyslogDebug to control which messages from the slurmctld and
    slurmd daemons get written to syslog.
 -- Add slurmdbd.conf configuration parameter DebugLevelSyslog to
    control which messages from the slurmdbd daemon get written to syslog.
bug 3933

Bug 3865

Adds assocation and QOS limits for the pack job as a whole

Clear a job's "wait reason" value of BeginTime" after that time has
passed. Previously a readon of "BeginTime" could be reported long
after the job's requested begin time had passed (for so long as the
current reason is "None".

Add support to sched/backfill for concurrent allocation of all pack job
      components including support of --time-min option.

Bug 3953

Pretty much fix the entire purpose of this max_agent_queue.

Bug 3159

Bug 3956

Bug 3967

step wasn't always gathered correctly.

Bug 3531

Fix for possible slurmctld abort with use of salloc/sbatch/srun
    --gres-flags=enforce-binding option.
bug 4008

Update from commit b40bd8d3

By removing the real locks we can get into a race condition where the prolog
starts and finishes before we get here and then we end up waiting forever.

Making the mutex a static seemed to help in many cases, but didn't
completely close the window.  Changing slurm_cond_wait to
slurm_cond_timedwait fixed the scenario where we would hit the window, but
not degrade performance the original commit provides.

There were also spots where if the job or step didn't exist it wouldn't
signal the conditional also providing a spot this could get stuck not
starting the job.

Fix regression from commit 52ce3ff0

Bug 3977

Avoid interleaving labels and output from various components of
   a pack job

Major re-write of task state container logic to support for list of
  containers rather than one container per srun command.

Modify all daemons to re-open log files on receipt of SIGUSR2 signal.  This
    is much than using SIGHUP to re-read the configuration file and rebuild
    various tables.
bug 3070

This is a regression from commit fec995e0.

It turns out using tok here was erroneous for situations where the gres had
a type and name and potentially a count (i.e. network:gigabit:1)

_get_gres_req_cnt() would alter the incoming char *config which is what tok
was.  So when we print it back to the requested string it would only have
what was there to the first ':'.  As we didn't need to \0 out the first char
as we skip over it anyway I just kept track of what the replaced \0 was for
the number portion and put it back when we are done copying it.

Related to bug 3521

No changes to logic

bug 3979