Bug 2130

Move slurmctld mail handler to separate thread for improved performance.
Original logic did fork/exec without separate thread and if the
slurmctld memory size is huge, then the time required for fork()
to complete can be significant.
bug 2252

Avoid hard-link/copy of script/environment files for job arrays. Use the
    master job record file for all tasks of the job array.
back port of commit 41eead64
bug 2220

Refactor logic to synchronize active bach jobs and their script/environment
    files, reducing overhead dramatically for large numbers of active jobs.
backport of commit 41eead64
bug 2220

If a job can never be started due to unsatisfied job dependencies, report
    the full original job dependency specification rather than the dependencies
    remaining to be satisfied (typically NULL).
bug 2197

in use.

_slurm_rpc_complete_job_allocation.

This is a rewrite of 438365ec which didn't catch the job_ptr wasn't
in a lock so the memory issue could still of existed.  This hopefully fixes
all the spots the job_ptr wasn't in the lock.

Fixes bug 2146

Bug 2261

15.08.5.

Bug 2243

bug 2171

Decrease parallelism in job cancel request to prevent denial of service
    when cancelling huge numbers of jobs.
bug 2256

Prevent triggering gang scheduling within a partition if configured with
    PreemptType=partition_prio and PreemptMode=suspend,gang.
The essence of this fix is to change a "<=" to "<" in cons_res/job_test.c:
-               if ((p_ptr->part_ptr->priority <= jp_ptr->part_ptr->priority) &&
+               if ((p_ptr->part_ptr->priority < jp_ptr->part_ptr->priority) &&
but logic was also added to insure that a partition configuration with
PreemptMode did not override PreemptType != partition_prio.
bug 2232

Previously an error() would be logged when the attempt to open the job
script using the new directory format failed but the successive fallback to the
old directory structure was successful, leading to confusion when troubleshooting.

Move emitted warnings to debug(), and only error() after failing to open in both
directory structures. Add a note about backwards compatibility to both functions -
we cannot remove these fallbacks as directory structure for pending jobs does not
change on Slurm version update, and people may need to chain multiple version
update together to get to a current slurm version which would correctly update
slurmctld state files but leave pending jobs in the old directory structure.

Bug #2244.

If a job is requeued while in the process of being launch, remove it's
    job ID from slurmd's record of active jobs in order to avoid generating a
    duplicate job ID error when launched for the second time (which would
    drain the node).
bug 2240

In slurmctld log file, log duplicate job ID found by slurmd. Previously was
    being logged as prolog/epilog failure.
bug 2240

so we can set it correctly before tasks are set.

news

through sacctmgr.

select/cray: Prevent NHC from running more than once per job or step.
bug 2192

In both sched/basic and backfill: If a job can not be started due to some
    account/qos limit, then don't start other jobs which could delay jobs. The
    old logic would skip the job and start other jobs, which could delay the
    higher priority job.
bug 2129

requests no time limit.

http://bugs.schedmd.com/show_bug.cgi?id=2177

Usernames are comma separated, not colon delimited. Bug #2222.
While here fix a few spelling mistakes.

Bug 2130

Adopted processes didn't have access to the job's devices.

Bug 2130

Full revert of c2fbf88f, 13b64c35 had caught part of this, but
this will revert it completely.  The code just wasn't needed in modern
Slurm.  It appears the patch came from an older version of Slurm that
didn't handle this correctly.

This reverts commit 29f25688.

Conflicts:
	NEWS

Looks like this isn't needed, commit c2fbf88f doesn't appear to be needed
and is what is causing this issue.  c2fbf88f was added from an older
version of Slurm where this was already handled correctly in commit
815e5a44.

Cray job NHC delayed until after burst buffer released and epilog completes
    on all allocated nodes.
bugs 2099 and 2192

Release a job's allocated licenses only after epilog runs on all nodes
    rather than at start of termination process.
bug 2192

sched/backfill - Delay backfill scheduler for completing jobs only if
    CompleteWait configuration parameter is set (make code match documentation).

Bug 2030

correctly when PriorityFlags=CALCULATE_RUNNING is set.

Previously the slurmctld could seg fault if the tres_alloc_str is NULL.

Prevent slurmdbd divide by zero if no associations defined at rollup time.

else behind.