Generate burst buffer use completion email immediately afer teardown
    completes rather than at job purge time (likely minutes later).
bug 2539

Change burst buffer use completion message from
    "SLURM Job_id=1360353 Name=tmp Staged Out, StageOut time 00:01:47" to
    "SLURM Job_id=1360353 Name=tmp StageOut/Teardown time 00:01:47"

Bug 2543.

on only one port like TopologyParam=NoInAddrAny does for everything else.

There's no /proc on *BSD, and BSD handles OOM in a completely different way.

Return [0-100:2] formatting, rather than [0,2,4,6,8,...] when using
a step function.

Was inadvertantly broken in 14.11 with commit 5ffdca92.

Bug 2535.

Fix Cray NHC spawning on job requeue. Previous logic would leave nodes
allocated to a requeued job as non-usable on job termination.

Specifically, each job has a "cleaning/cleaned" flag. Once a job
terminates, the cleaning flag is set, then after the job node health
check completes, the value gets set to cleaned. If the job is requeued,
on its second (or subsequent) termination, the select/cray plugin
is called to launch the NHC. The plugin sees the "cleaned" flag
already set, it then logs:
error: select_p_job_fini: Cleaned flag already set for job 1283858, this should never happen
and returns, never launching the NHC. Since the termination of the
job NHC triggers releasing job resources (CPUs, memory, and GRES),
those resources are never released for use by other jobs.

Bug 2384

An error in slurmconfgen_smw.py caused it to parse the nic as the nid.
On some systems those values differ, causing the generated slurm.conf file to
be incorrect.

Bug 2532.

route_p_split_hostlist was not thread-safe, and would cause
one of several segfaults depending on where in the initialization
code each thread was.

Bug 2495.

Was incorrectly displaying "(null)" even when loaded successfully.

Bug 2507

Step GRES value changed from type "int" to "int64_t" to support larger
values. Previous logic could fail in step allocation values over 32-bits.
Other GRES values are 64-bit.

slurmstepd to close potential open ones.

It was pointed out the slurmd using acct_gather_energy/ipmi links to
freeipmi which could possibly open /dev/ipmi0 without the close on exec
flag set as root while launching a step leaving it open in the users app.

What this does is sets the flag on the first 256 to mitigate the concern.

Reported by Maksym Planeta.

Bug 2506

Previous logic tested whatever the job's partition pointer indicated
rather than the partition we are trying to run the job in. This bug
was introduced in Slurm version 15.08.5, Nov 16, 2015, commit
94f0e948
bug 2499

Insure that a job is completely launched before trying to suspend it.
Previous logic would start suspend logic early in the life of the
slurmstepd process, after it's listening socket was open but before
the tasks were launched. This defers the suspend logic until after
all prologs and setup completes and the tasks are launched. This is
important in the case of gang scheduling, in which newly launched
jobs can be immediately suspended.
bug 2494

Add note to slurm.conf man page about setting "--cpu_bind=no" as part
of SallocDefaultCommand if a TaskPlugin is in use.

was also given.

a partition.

This also reverts most of commit fa331e30 as well as commit bd9fa830
which would try to set the pn_min_cpus every time a job was updated.
If a job didn't request node counts then they were hosed.

This commit takes away the magic which was screwing things up.  Now the
person gets what they asked for without magic changing things.

Bug 2302
Bug 2742
Bug 2478

erroneously.

This whole process could probably be done better by keeping track of
old values and new values and only calling one function instead of a
pre and post function, but that can probably wait for future generations
of the code as it works now and is probably adequate for the time being.

Bug 2352

BurstBuffer/cray - Defer job cancellation or time limit while "pre-run"
    operation in progress to avoid inconsistent state due to multiple calls
    to job termination functions.
bug 2454

a new account and making it a default all at once.

Bug 2428

Match acct_gather_energy/rapl plugin. Bug 2397.

Control whether the scheduler will continue to try to run jobs in
a partition if a higher priority job is stuck due to an association
limit.

Can cause starvation for larger jobs, but will improve throughput and
utilization for systems that have extensively divvyed up their resources
through association/QOS limits.

Bug 2388 and 2452.

Bug 2453

have been selected set the time limit appropriately if the job didn't
request one.

If the partition has no DefaultTime setting, and no time_limit was given
for the job, job_ptr->time_limit == NO_VAL.

With AccountingStorageEnforce=safe this will prevent jobs from ever
starting if the association has any limit set for CPUMins.
(NO_VAL * cpus is a very large number, but if no time_limit is given
anywhere that is what they get :))

Bug 2388.

a parsing option.

Dynamically set columns widths based on largest number in column.