of Slurm daemons.

If a QOS was added for the job and then removed and it just happened
to be the largest QOS id wise if the slurmctld was restarted and the job
wasn't flushed out yet it could mess things up.

accounting_storage/filetxt.

Logic introdiced in version 14.03.10 to support requeueing of jobs
with GRES allocated to currently running steps broke select/linear
due to differernces in the plugin logic. The commit with the bad
logic is 1209a664

Bug 1301

Fix jobs from starting in overlapping reservations that won't finish before a "maint" reservation begins.

Bug 1290

when the DBD is down.

Log Cray MPI job calling exit() without mpi_fini(), but do not treat it as
a fatal error. This partially reverts logic added in version 14.03.9.
bug 1171

better.

in BASIL was changed.

Double max string that Slurm can pack from 16MB to 32MB to support
larger MPI2 configurations.

This can happen if the specified job ID is not found.

Bug 1253

Bug 1255

Do not requeue a batch job from slurmd daemon if it is killed while in
the process of being launched (a race condition introduced in v14.03.9).
This partially reverts commit 2bc9bc29

with CR_PACK_NODES.

Really do commit d388dd67 a different way to get the same info and
be able to lay out tasks correctly when --hint=nomultithread.

tests on a 4 core 8 thread system are
srun -n6 --hint=nomultithread --exclusive whereami | sort -h
srun: cpu count 6
   0 snowflake0 - MASK:0x1
   1 snowflake0 - MASK:0x2
   2 snowflake0 - MASK:0x4
   3 snowflake0 - MASK:0x8
   4 snowflake1 - MASK:0x1
   5 snowflake1 - MASK:0x2

and

srun -n10 -N5 --hint=nomultithread --exclusive whereami | sort -h
srun: cpu count 10
   0 snowflake0 - MASK:0x1
   1 snowflake0 - MASK:0x2
   2 snowflake0 - MASK:0x4
   3 snowflake0 - MASK:0x8
   4 snowflake1 - MASK:0x1
   5 snowflake1 - MASK:0x2
   6 snowflake1 - MASK:0x4
   7 snowflake2 - MASK:0x1
   8 snowflake3 - MASK:0x1
   9 snowflake4 - MASK:0x1

    an maintenance reservation that is not active yet.

work "partition".

reference bug 1246

is requested.

This is a re-factor of commit e5635a76 related to bug 1148 to handle
the cases where a job could run, but an error was given when selecting
the nodes.

lock was locked outside of the function or not.  This also fixes a race
condition when adding a QOS and planning on using it right away when the
controller is busy with previous requests.

PerCPU.

Before it wasn't taking into account if the user was requesting per node
memory or the job was told it needed to use less than the node allowed.

This was an unrealized regression from commit 0da01963.  The problem
is we were clearing the job_ptr->job_resrcs too early.  This patch fixes
it to wait until the job is actually being requeued so it does the right
thing.

This isn't that big of an issue for 14.03, but 14.11 added more to this
string which could overflow the buffer since sprintf is used instead of
snprintf.  Using xstrfmtcat fixes the issue and is easier to read code.

amount of tasks / number of node.

are specified.

This is a fix to commit b9cc5b31 which just didn't know
mc_ptr->ntasks_per_core is initialized to INFINITE.  Without it the
node_cnt packed would be set to 1 on the user tools.

This fixes bug 1148.

We've seen slurmctld crashes due to negative job array indices.

BGQ: Fix race condition when job fails due to hardware failure and is
requeued. Previous code could result in slurmctld abort with NULL pointer.
bug 1096

Fix bug that prevented preservation of a job's GRES bitmap on slurmctld
restart or reconfigure (bug was introduced in 14.03.5 "Clear record of a
job's gres when requeued" and only applies when GRES mapped to specific
files).
bug 1192