with hwloc.

This patch has two parts:
1. When a job is intially submitted, the Slurm was failing to set
   an initial reason for the job not starting.
2. After a job was submitted, it was sometimes failing to reset
   the job's reason. It was also failing to reset the "last_job_update"
   time, so something like "squeue -i1" would not get the new reason.
bug 3025

Fix multipart srun submission with EnforcePartLimits=NO and job violating
    the partition limits.
bug 3025

bug 3011

both)  Continuation of commit 95fa8e8e.

If all GRES were not defined on all nodes OR if a regular expression was used
   for a GRES file configuration (e.g. in gres.conf
   "Type=gpu Files=/dev/nvidia[0-4]"), then memory corruption was likely.
   The logic has been bad since its inception several years ago.

POLLRDHUP does not exist on BSD, define to POLLHUP as done elsewhere.

The attached patch switches to a more reliable method of detecting
service nodes, using xtcli status. In addition, it switches to the print
function to be better compatible with python 3.

multiple versions.

on the most recent pmix_vX.so file.
One is free to directly use new "pmix_vX" plugin names as well.

NOTE: there is one small artifact introduced by this fix - there will be
no "pmix" name in the output of "srun --mpi=list" but the most recent plugin
will be duplicated:

$ ./srun --mpi=list
srun: MPI types are...
srun: mpi/none
srun: mpi/mvapich
srun: mpi/mpich1_shmem
srun: mpi/mpichgm
srun: mpi/pmix_v1      <----
srun: mpi/mpichmx
srun: mpi/lam
srun: mpi/pmi2
srun: mpi/openmpi
srun: mpi/mpich1_p4
srun: mpi/pmix_v1      <----

conflicting pmix versions.

linking against.

commit 20a73c508e1.

of what is in the header file of pmix.

To ease the distribution process, plugin names will be automatically
adjusted to identify the version of API that it can support,
ie: pmix_v1 and pmix_v2.
This provides the ability for distro's to create separate
non-conflicting packages for each API generation.

Bug 2986

Insure reported expected job start time is not in the past for pending jobs.
bug 3002

burst_buffer/cray: Requeue, but do not hold a job which fails the pre_run
    operation.
bug 3009

Bug 2984.

Only mark job_id as zero for batch step (when all job steps would be
cleared), not for individual steps which prevented successive steps
from being cancelled.

Bug 2984.

slurmstepd modified to pre-load all relevant plugins at startup to avoid
    the possibility of modified plugins later resulting in inconsistent API
    or data structures and a failure of slurmstepd.
bug 2334

bug 2655

Add more details to the log messages for improved ability to
diagnose problems.

Bug 2983.

Continuation of 18947480. Commit cff2effc made cpu_freq_init
unconditionally set up the structures, so this reinitialization was
discarding the true original info, which prevents the frequency
and governor from being restored properly after the step completes.

Don't about step launch if job reaches expected end time while node is
    configuring/booting (NOTE: The job end time will be adjusted after node
    becomes ready for use).
bug 2985