If the backup dbd happened to be doing rollup at the time the primary resumed
both the primary and the backup would be doing rollups and causing contention on
the database tables. The backup would wait for the rollup handler to finish
before giving up control.

The fix is to cancel the rollup_handler and let the backup begin to shutdown so
that it will close an existing connections and then re-exec itself. The re-exec
helps because the rollup handler spawns a thread for each cluster to rollup and
just cancelling the rollup handler doesn't cancel the spawned threads from the
rollup handler. This cleans up the dbd and locks. The re-exec only happens in
the backup if the primary resumed and a rollup was happening.

Bug 1988

Fix case where if the backup slurmdbd has existing connections when it gives up control that the it would be killed.

If the backup had existing connections when giving up control, it would try to
signal the existing threads by using pthread_kill to send SIGKILL to the
threads. The problem is that SIGKILL doesn't go the thread but the main process
and the backup dbd would be killed.

when a cold-start (-c) happens to the slurmctld.

This was intended as a step toward managing jobs across mutliple
  clusters, but we will be pursuing a very different design.

from a user.

This would cause the slurmctld to cache the old default which wasn't valid
and cause the user to have to request the association always.

byg 2013

database but the start record hadn't made it yet.

Add acct_gather_energy/ibmaem plugin for systems with IBM Systems Director
    Active Energy Manager.

requirements.

Add acct_gather_energy/ibmaem plugin for systems with IBM Systems Director
    Active Energy Manager.

bug 2011

','.

bug 1999

Don't requeue RPC going out from slurmctld to DOWN nodes (can generate
    repeating communication errors).
bug 2002

This will only happen if a PING RPC for the node is already queued
  when the decision is made to power it down, then fails to get
  a response for the ping (since the node is already down).
bug 1995

If a job's CPUs/task ratio is increased due to configured MaxMemPerCPU,
then increase it's allocated CPU count in order to enforce CPU limits.
Previous logic would increase/set the cpus_per_task as needed if a
job's --mem-per-cpu was above the configured MaxMemPerCPU, but NOT
increase the min_cpus or max_cpus varilable. This resulted in allocating
the wrong CPU count.

This will only happen if a PING RPC for the node is already queued
  when the decision is made to power it down, then fails to get
  a response for the ping (since the node is already down).
bug 1995

values.

This required a fairly major re-write of the select plugin logic
bug 1975

Correct some cgroup paths ("step_batch" vs. "step_4294967294", "step_exter"
    vs. "step_extern", and "step_extern" vs. "step_4294967295").

If a job's CPUs/task ratio is increased due to configured MaxMemPerCPU,
then increase it's allocated CPU count in order to enforce CPU limits.
Previous logic would increase/set the cpus_per_task as needed if a
job's --mem-per-cpu was above the configured MaxMemPerCPU, but NOT
increase the min_cpus or max_cpus varilable. This resulted in allocating
the wrong CPU count.

Requeue/hold batch job launch request if job already running. This is
  possible if node went to DOWN state, but jobs remained active.
In addition, if a prolog/epilog failed DRAIN the node rather than
  setting it down, which could kill jobs that could continue to
  run.
bug 1985

Bug 1938

Bug 1984

When nodes have been allocated to a job and then released by the
  job while resizing, this patch prevents the nodes from continuing
  to appear allocated and unavailable to other jobs. Requires
  exclusive node allocation to trigger. This prevents the previously
  reported failure, but a proper fix will be quite complex and
  delayed to the next major release of Slurm (v 16.05).
bug 1851

When nodes have been allocated to a job and then released by the
  job while resizing, this patch prevents the nodes from continuing
  to appear allocated and unavailable to other jobs. Requires
  exclusive node allocation to trigger. This prevents the previously
  reported failure, but a proper fix will be quite complex and
  delayed to the next major release of Slurm (v 16.05).
bug 1851

Add mail wrapper script "smail" that will include job statistics in email
    notification messages.
bug 1611

Add ability to change a job array's maximum running task count:
    "scontrol update jobid=# arraytaskthrottle=#"
bug 1863

option.