Add a new configuration parameter MinRAMSpace which sets a lower bound on
memory.limit_in_bytes and memory.memsw.limit_in_bytes . This is required in
case an administrator or user sets an absurdly low value for memory limit,
potentially causing the slurmstepd to be terminated by the OOM killer.

MinRAMSpace is set in MB of RAM and is 30 by default. (An arbitrarily
chosen value)

The use of whole percent values for cgroup.conf parameters such
as AllowedRAMSpace, MaxRAMPercent, AllowedSwapSpace and MaxSwapPercent
may be too coarse grained on systems with large amounts of memory.
(e.g. 1% of 64G is over 650MB).

This patch allows these percentage values to be arbitrary floating
point numbers to allow finer grained tuning of these limits and
parameters.

Treat a 0 byte memory limit from SLURM as unlimited and instead use
MaxRAMPercent and MaxSwapPercent as RAM and Swap limits for the job/job
step. This avoids creating a memory cgroup with limit_in_bytes = 0,
which would end up causing the cgroup to OOM before slurmstepd could
even be started.

This also allows systems in which SLURM isn't explicitly allocating
memory to use the task/cgroup plugin with ConstrainRAMSpace=yes.

Calculate the upper bound RAM in bytes and Swap in bytes that may
be used by any one cgroup and apply this limit in the task/cgroup
code.

As a failsafe we may want to put a hard limit on memory.limit_in_bytes
and memory.memsw.limit_in_bytes when using cgroups. This patch adds
MaxRAMPercent and MaxSwapPercent which are taken as percentages of
available RAM (RealMemory as reported by slurmd), and which will be
applied as upper bounds when creating memory controller cgroups.

Add conf->real_memory_size to the list of slurmd_conf_t members that
are propagated to slurmstepd during a job step launch. This makes the
amount of RAM available on the system (as determined by slurmd) available
for use in slurmstepd plugins or slurmstepd itself, without having to
recalculate its value.

There was some duplicated code in task_cgroup_memory_create. In order
to facilitate extending this code in the future, refactor it into
a common function memcg_initialize().

The example cgroup release agent packaged and installed with
SLURM assumes a base directory of /cgroup for all mounted
subsystems. Since the mount point is now configurable in SLURM,
this script needs to be augmented to determine the location
of the subsystem mount point at runtime.

cgroups code currently assumes cgroup subsystems will be mounted
under /cgroup, which is not the ideal location for many situations.
Add a new cgroup.conf parameter to redefine the mount point to an
arbitrary location. (for example, some systems may already have
cgroupfs mounted under /dev/cgroup or /sys/fs/cgroup)

Prevent slurmctld crashing with divide by zero with a configuration of MaxMemPerCPU=0.

block happens correctly now.

Major re-write of the CPU Management User and Administrator Guide (web
page) by Martin Perry, Bull.

PluginDir is a path. It shouldn't be an error to have duplicate
plugins in your path. Plus, the error is not helpful because it
doesn't specify which path is not being loaded. Therefore, just
remove the error and load the first plugin in the path as expected.

Fix bugs in sched/backfill with respect to QOS reservation support and job
time limits. Patch from Alejandro Lucero Palau (Barcelona Supercomputer Center).

Fix to GRES allocation logic when resources are associated with specific
CPUs on a node. Patch from Steve Trofinoff, CSCS.

(i.e. 1-9,0 instead of 0-9).  The bug would cause 'sacct -N nodename' to
not give correct results on these systems.

is in an error state, won't deny jobs.

restarts of the slurmctld.

admin sets the state to error.

Advise use of the logrotate tool in order to avoid SLURM log files
from growing too large. Patch from Rod Shultz, Bull.

Do not treat the absence of a gres.conf file as a fatal error on systems
configured with GRES, but set GRES counts to zero. These counts can be
Counts can be altered by node_config_load() in the gres plugin.

The slurmctld code that processes job notify messages unecessarily
restricts these messages to be from the slurm user or root. This
patch allows users to send notifications to their own jobs.

Many cosmetic modifications to eliminate warning message from GCC version
4.6 compiler, mostly due to unused variables.

jobs happen to be running on blocks not in the new config.

salloc/mpirun does not play well together with task affinity socket binding.  The following example illustrates the problem.

[sulu] (slurm) mnp> salloc -p bones-only -N1-1 -n3 --cpu_bind=socket mpirun cat /proc/self/status | grep Cpus_allowed_list
salloc: Granted job allocation 387
--------------------------------------------------------------------------
An invalid physical processor id was returned ...

The problem is that with mpirun jobs Slurm launches only a single task, regardless of the value of -n. This confuses the socket binding logic in task affinity.  The result is that task affinity binds the task to only a single cpu, instead of all the allocated cpus on the socket.  When mpi attempts to bind to any of the other allocated cpus on the socket, it gets the "invalid physical processor id" error. Note that the problem may occur even if socket binding is not explicitly requested by the user.  If task/affinity is configured and the allocated CPUs are a whole number of sockets, Slurm will use "implicit auto binding" to sockets, triggering the problem.
Patch from Martin Perry (Bull).

Update reservation web page to describe mechanism to reserve CPUs rather than whole nodes and provide an example.

Avoid clearing a job's reason from JobHeldAdmin or JobHeldUser when it is
otherwise updated using scontrol or sview commands. Patch based upon work
by Phil Eckert (LLNL).