Prevent a heterogeneous job allocation from including the same nodes in
      multiple components (required by MPI jobs spanning components).

Make error specify the same option name as on srun input

Check for an invalid pack group value before checking for nothing
  to start

Parallel "cp -f src dst" could fail with file exists if run more
  than once at the same time on the same file. Replace wtih
  "rm; sleep 5; cp"

Set SLURM_JOB_CPUS_PER_NODE and SLURM_NODE_ALIASES environment
  variables in srun for pack jobs to reflect the resources used
  by that specific srun (select pack groups).

This applies to job steps for MPI. Even if there is more than one
  pack job component in a single MPI_COMM_WORLD, they will share
  a common SLURM_JOBID.

Do file copy with "-f" (force) option

This sets more per-pack-job environment variables for launched steps.
All of the following are used by Open MPI:
SLURM_CPUS_PER_TASK
SLURM_STEP_NUM_TASKS
SLURM_TASKS_PER_NODE
A few more env vars are still needed by OpenMPI

Test was sporadically failing on smd without sleep

SelectType does not need to include "memory" with task affinity
 or cgroup to cancel job upon reaching specified memory limit

The pack job logic now includes a job ID in the message, which
  necessitated changes to the test in order to prevent a failure.

Without this patch, the default name of all pack job components
  submitted by salloc was "sh"

These are required by OpenMPI

Commit c125875f introduced a
  change that with some compilers generates a warning about the
  "const" variable definition being ignored on an assignment.

Modify output of "--mpi=list" to avoid duplicates for version numbers in
    mpi/pmix plugin names.

Notify user if he tries to start a job step with a higher pack group
  value than the number available in the job allocation. Previous
  logic would abort the srun.

Modify sbcast command and srun's --bcast option to support heterogeneous
      jobs.
bug 4099

This should have been addressed in earlier commit, but was not noticed
  until trying to build on cluster with nrt software

Make sure no vestigial jobs are left after the test

In srun, if only some steps are allocated and one step allocation fails,
 then delete all allocated steps.

Set SLURM_NTASKS environment variable to reflect global task count
(needed by MPI).

Set SLURM_PROCID environment variable to reflect global task rank
(needed by MPI).

Coverity CID 45205

Coverity CID 44959

Coverity CID 171461