Make sure that old step data is purged when a job is requeued.
Without this logic, if a job terminates abnormally then old step
data may be left in slurmctld. If the job is then requeued and
started on a different node, referencing that old job step data
can result in abnormal events. One specific failure mode is if
the job is requeued on a node with a different number of cores,
and the step terminated RPC arrives later, the job and step
bitmaps of allocated cores can differ in size generating an
abort.
bug 1660

Bug 1657

Bug 1618

cpu count.

Add re-entrant versions of glibc time functions (e.g. localtime) to Slurm
in order to eliminate rare deadlock of slurmstepd fork and exec calls.
bug 1638

utilization.

random crashing in db2 when the slurmctld is exiting.

Signed-off-by: Danny Auble <da@schedmd.com>

In slurmctld communication agent, make the thread timeout be the configured
value of MessageTimeout (or 30 seconds, whichever is larger) rather than
30 seconds.

Fix scancel bug which could return an error on attempt to signal a job step.
A simple "scancel 12.3" to signal a specific job step would fail. Adding
another option (say "-i", "--partion=", etc.) would fix this.

before ending the job.

will make it so the slurmctld will not signal the apid's in a batch job.
Instead it relies on the rpc coming from the slurmctld to kill the job to
end things correctly.

Refactor scancel so that all pending jobs are cancelled before starting
cancellation of running jobs. Otherwise they happen in parallel and the
pending jobs can be scheduled on resources as the running jobs are being
cancelled.

Add SchedulerParameters option of sched_min_interval that controls the
minimum time interval between any job scheduling action. The default value
is zero (disabled).
bug 1623

Fix some power_save mode issues: Parsing of SuspendTime in slurm.conf was
bad, powered down nodes would get set non-responding if there was an
in-flight message, and permit nodes to be powered down from any state.
bug 1619

Initialize some variables used with the srun --no-alloc option that may
cause random failures.

Also see:
https://github.com/asanchez1987/jobcomp-elasticsearch

job.  This is already handled on the stepd when the script finishes.

often an inventory request is handled.

free afterwards would not have zeroed out memory on the variables that
didn't get unpacked.

sbatch to stop parsing script for "#SBATCH" directives after first command.
It keeps parsing so long as lines contain only white space or comments
(first non-white space character is '#').