burst_buffer/cray - Do not execute "pre_run" operation until after all nodes
    are booted and ready for use.
bug 3461

partitions.

Insure job does not start running before node is booted and PrologSlurmctld
    is complete.
bug 3446

Ported from 7a4aa7f2.

burst_buffer/cray - Support default pool which is not the first pool
    reported by DataWarp and log in Slurm when pools that are added or removed
    from DataWarp.
bug 3453

This reverts commit fd690a9c.

was there for jobs and steps, but for no real reason.  I couldn't see a
reason anyway.

Jobs preempted with PreemptMode=REQUEUE were incorrectly recorded as
REQUEUED in the accounting.

Bug 3444

bug 3448

Bug 3447

Bug 3444

Fix srun I/O race condtion to eliminate a error message that might be
    generated if the application exits with outstanding stdin.
bug 3166

Properly set SLURM_JOB_GPUS environment variable for Prolog.
bug 3437

Clear job's reason of "BeginTime" in a more timely fashion and/or prevents
    them from being stuck in a PENDING state. There are multiple ways of
    clearing the reason, especially on a lightly loaded system, but the
    state can persist indefinitely on a heavily loaded system.
bug 3368

Fix to logic for getting expected start time of existing job ID with
    explicit begin time that is in the past. Previous logic would
    compare that (past) begin time with advanced reservations that
    would compete with it rather than the current time.

CRAY systems only: TaskPlugins must list task/cgroup before task/cray in
    order for the cgroup files to be created before task/cray runs. Without
    this change, the task/cray plugin frequently produces errors about the
    "mems" file being missing. The errors don't seem consistent, so this
    probably involves a race condition. Note that NERSC uses this order
    today and I changed read_config.c to produce a fatal error if the order
    is reversed.

Turns out this never worked, ever.  What used to happen is if the protocol_version that was
read in didn't match the rpc_version given to unpack things was just 0.  What this does
now is set the rpc_version to what was stored making it all good.

Revert logic originally added for bug 3166.
  Revisit as time permits.
bug 3166

Bug 3431

burst_buffer/cray - Fix race condition that could cause multiple batch job
    launch requests resulting in downed nodes.
bug 3366

This reverts commit b9bff82f.

Bug 1599

slurmctld/agent race condition fix: Prevent job launch while PrologSlurmctld
    daemon is running or node boot in progress.
bug 3366

If a lower version client would try to communicate with a higher version
controller the dbd would return the controller's version and the client
would use that version to talk to the controller. When the controller
would respond, the client wouldn't know how to unpack the higher version
msg.

Bug 2508