task/cray's _get_numa_nodes() function needs to run before task/cgroup
cleans up the cgroup hierarchies, otherwise ALPS memory compaction will
never run.

Also move task_p_add_pid() outside the #ifdef HAVE_NATIVE_CRAY
block so that the plugin will load (albeit without any functionality)
on non-Cray systems for testing purposes.

Revise documentation and provided slurm.conf templates as well.

Bug 3154.

_select_nodes_parts() was resetting state_reason to an admin hold
without regard to admin vs user hold state. state_reason is the only
place that user vs. admin is distinguished, so this prevented users
from releasing these jobs.

Bug introduced by commit fb46c84b in 16.05.5.

Bug 3197.

This is an addition to commit cb7ed937

bug 3191

Improve reported estimates of start and end times for pending jobs.
bug 3184

Cray: Prevent abort in backfill scheduling logic for requeued job that has
    been cancelled while NHC is running.
bug 3185

new glibc 2.24+ that depricates readdir_r.

Added node_features/knl_generic plugin for KNL support on non-Cray systems.
NOTE: This plugin is still under development.

Do not propagate SLURM_UMASK environment variable to batch script.
bug 2609

Correct a bitmap test function (used only by the select/bluegene plugin).
  The effect of this bug is probably very limited as it will in almost
  all cases revert prematurely to a bit-by-bit test rather than using
  a full-word test.
bug 3145

Cannot use ClusterName without reading a config file
that may not exist.

Bug 3026.

This introduced an inadvertent dependency on the config file,
which does not exist when setting up a new cluster.

Bug 3026.

This reverts commit c39f9ac9.

task/affinity plugin: Honor a job's --ntasks-per-socket and
    --ntasks-per-core options in task binding.
bug 3118

Preserve non-KNL node features when updating the KNL node features for a
    multi-node job in which the non-KNL node features vary by node.

node_features/knl_cray plugin: If the reconfiguration of nodes for an
    interactive job fails, kill the job (it can't be requeued like a batch job).

node_features/knl_cray plugin: Add separate thread to interact with capmc
    in response to unexpected node reboots.
bug 3153

bug 3091

Prevent possible divide by zero in select/cons_res if a node's board count
    is higher than it's socket count.
bug 3155

If a node's socket or core count are changed at registration time (e.g. a
    KNL node's NUMA mode is changed), change it's board count to match.
bug 3155

Cray: The slurmd can manipulate the socket/core/thread values reported based
    upon the configuration. The logic failed to consider select/cray with
    SelectTypeParameters=other_cons_res as equivalent to select/cons_res.
bug 3155

abs() should not be used on long long variables as it would
truncate if strictly confirming to C99. Use llabs() instead.

Fix to commit 2aefc66b.

always just AccountingPolicy.

configure.

Bug 3124.

node_features plugin - Add "mode" argument to node_features_p_node_xlate()
    function to fix some bugs updating a node's features using the node update
    RPC. Without this change it is impossible to clear the active features of
    a node or reset non-KNL node features.

node_features/knl_cray plugin: Substantially streamline and speed up logic
    to load current node state on reconfigure failure or unexpected node boot.
    Completely eliminate capmc calls and just use cnselect to load current
    node mode information.

node_features/knl_cray plugin: drain any node not reported by
    "capmc node_status" on startup or reconfig. Also re-tests
    on failed node restart for job.

node_features/knl_cray plugin: Remove any KNL MCDRAM or NUMA features from
    node's configuration if capmc does NOT report the node as being KNL.
    For example, we don't want a non-KNL node with features="quad,cache".

Add new knl.conf configuration parameter CapmcRetries
Modify capmc_suspend and capmc_resume to retry operations when
  Cray State Manager is down.
Add retry logic to node_features/knl_cray to handle Cray State
  manager being down.
bug 3100

Otherwise they'll truncate when packed into the RPC and end up
as some bizarre value at the controller.

Bug 3098.

Set completed time for pending/running runaway jobs to the max of
(start, eligible, submit) times.

Bug 3075

Avoid using slurm_forward_data because it causes thread spawn that
introduces unwanted delays.

Bug 3102.

Also correct the value of NICE_OFFSET used within the perl API.

Bug 3098.

Bug 3051.