time.

static/overlap systems when some hardware issue happens when restarting
the slurmctld.

a dynamic system and mark the block in error on a static/overlap system.

Bug 2273

Don't use lower weight nodes for job allocation when topology/tree used.
bug 2284

Preemption/gang scheduling: If a job is suspended at slurmctld restart or
    reconfiguration time, then leave it suspended rather than resume+suspend.
bug 2274

task/affinity: Disable core-level task binding if more CPUs required than
    available cores.
bug 2267

Log as error if more than 3 aeld connects per second that cause is
  likely duplicate slurmctld daemon
bug 2278

This is needed to properly enforce limits and account for usage.

Remove the 1024-character limit on lines in batch scripts, which was
causing long lines to be silently truncated. I noticed it when jobs were
getting created with fewer dependencies than specified.

Also increase the line length when showing job info.

sched/backfill: If a job can not be started within the configured
    backfill_window, set it's start time to 0 (unknown) rather than the end
    of the backfill_window.
bug 2100

If a pending job array has multiple reasons for being in a pending state,
    then print all reasons in a comma separated list.

Before:
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
       97354_[1-4]     debug      tmp    jette PD       0:00      1 (Resources)

After:
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
       97354_[1-4]     debug      tmp    jette PD       0:00      1 (Resources,JobHeldUser)

Bug 2130

Move slurmctld mail handler to separate thread for improved performance.
Original logic did fork/exec without separate thread and if the
slurmctld memory size is huge, then the time required for fork()
to complete can be significant.
bug 2252

Avoid hard-link/copy of script/environment files for job arrays. Use the
    master job record file for all tasks of the job array.
back port of commit 41eead64
bug 2220

Refactor logic to synchronize active bach jobs and their script/environment
    files, reducing overhead dramatically for large numbers of active jobs.
backport of commit 41eead64
bug 2220

If a job can never be started due to unsatisfied job dependencies, report
    the full original job dependency specification rather than the dependencies
    remaining to be satisfied (typically NULL).
bug 2197

in use.

_slurm_rpc_complete_job_allocation.

This is a rewrite of 438365ec which didn't catch the job_ptr wasn't
in a lock so the memory issue could still of existed.  This hopefully fixes
all the spots the job_ptr wasn't in the lock.

Fixes bug 2146

Bug 2261

15.08.5.

Bug 2243

bug 2171

Decrease parallelism in job cancel request to prevent denial of service
    when cancelling huge numbers of jobs.
bug 2256

Prevent triggering gang scheduling within a partition if configured with
    PreemptType=partition_prio and PreemptMode=suspend,gang.
The essence of this fix is to change a "<=" to "<" in cons_res/job_test.c:
-               if ((p_ptr->part_ptr->priority <= jp_ptr->part_ptr->priority) &&
+               if ((p_ptr->part_ptr->priority < jp_ptr->part_ptr->priority) &&
but logic was also added to insure that a partition configuration with
PreemptMode did not override PreemptType != partition_prio.
bug 2232

Previously an error() would be logged when the attempt to open the job
script using the new directory format failed but the successive fallback to the
old directory structure was successful, leading to confusion when troubleshooting.

Move emitted warnings to debug(), and only error() after failing to open in both
directory structures. Add a note about backwards compatibility to both functions -
we cannot remove these fallbacks as directory structure for pending jobs does not
change on Slurm version update, and people may need to chain multiple version
update together to get to a current slurm version which would correctly update
slurmctld state files but leave pending jobs in the old directory structure.

Bug #2244.

If a job is requeued while in the process of being launch, remove it's
    job ID from slurmd's record of active jobs in order to avoid generating a
    duplicate job ID error when launched for the second time (which would
    drain the node).
bug 2240

In slurmctld log file, log duplicate job ID found by slurmd. Previously was
    being logged as prolog/epilog failure.
bug 2240

so we can set it correctly before tasks are set.

news

through sacctmgr.

select/cray: Prevent NHC from running more than once per job or step.
bug 2192

In both sched/basic and backfill: If a job can not be started due to some
    account/qos limit, then don't start other jobs which could delay jobs. The
    old logic would skip the job and start other jobs, which could delay the
    higher priority job.
bug 2129

requests no time limit.

http://bugs.schedmd.com/show_bug.cgi?id=2177

Usernames are comma separated, not colon delimited. Bug #2222.
While here fix a few spelling mistakes.