RELEASE NOTES FOR SLURM VERSION 16.05
5 January 2016

IMPORTANT NOTES:
ANY JOBS WITH A JOB ID ABOVE 2,147,463,647 WILL BE PURGED WHEN SLURM IS
UPGRADED FROM AN OLDER VERSION! Reduce your configured MaxJobID value as needed
prior to upgrading in order to eliminate these jobs.

If using the slurmdbd (Slurm DataBase Daemon) you must update this first.
The 15.08 slurmdbd will work with Slurm daemons of version 14.03 and above.
You will not need to update all clusters at the same time, but it is very
important to update slurmdbd first and having it running before updating
any other clusters making use of it.  No real harm will come from updating
your systems before the slurmdbd, but they will not talk to each other
until you do.  Also at least the first time running the slurmdbd you need to
make sure your my.cnf file has innodb_buffer_pool_size equal to at least 64M.
You can accomplish this by adding the line

innodb_buffer_pool_size=64M

under the [mysqld] reference in the my.cnf file and restarting the mysqld. The
buffer pool size must be smaller than the size of the MySQL tmpdir. This is
needed when converting large tables over to the new database schema.

Slurm can be upgraded from version 14.03 or 14.11 to version 15.08 without loss
of jobs or other state information. Upgrading directly from an earlier version
of Slurm will result in loss of state information.

If using SPANK plugins that use the Slurm APIs, they should be recompiled when
upgrading Slurm to a new major release.


HIGHLIGHTS
==========
 -- Implemented and documented PMIX protocol which is used to bootstrap an
    MPI job. PMIX is an alternative to PMI and PMI2.
 -- Change default CgroupMountpoint (in cgroup.conf) from "/cgroup" to
    "/sys/fs/cgroup" to match current standard.
 -- Add Multi-Category Security (MCS) infrastructure to permit nodes to be bound
    to specific users or groups.
 -- Added --deadline option to salloc, sbatch and srun. Jobs which can not be
    completed by the user specified deadline will be terminated with a state of
    "Deadline" or "DL".
 -- Add an "scontrol top <jobid>" command to re-order the priorities of a user's
    pending jobs. May be disabled with the "disable_user_top" option in the
    SchedulerParameters configuration parameter.
 -- Cray: Not running the Node Health Check after every job and step is now the
    default. Configure SelectTypeParameters with the NHC and/or NHC_STEP to
    run them.

RPMBUILD CHANGES
================
 -- Remove all *.la files from RPMs.
 -- Implemented the --without=package option for configure.

CONFIGURATION FILE CHANGES (see man appropriate man page for details)
=====================================================================
 -- Change default CgroupMountpoint (in cgroup.conf) from "/cgroup" to
    "/sys/fs/cgroup" to match current standard.
 -- Introduce a new parameter "requeue_setup_env_fail" in SchedulerParameters.
    If set, a job that fails to setup the environment will be requeued and the
    node drained.
 -- The partition-specific SelectTypeParameters parameter can now be used to
    change the memory allocation tracking specification in the global
    SelectTypeParameters configuration parameter. Supported partition-specific
    values are CR_Core, CR_Core_Memory, CR_Socket and CR_Socket_Memory. If the
    global SelectTypeParameters value includes memory allocation management and
    the partition-specific value does not, then memory allocation management for
    that partition will NOT be supported (i.e. memory can be over-allocated).
    Likewise the global SelectTypeParameters might not include memory management
    while the partition-specific value does.

COMMAND CHANGES (see man pages for details)
===========================================
 -- sbatch to read OpenLava/LSF/#BSUB options from the batch script.
 -- Add sbatch "--wait" option that waits for job completion before exiting.
    Exit code will match that of spawned job.
 -- Job output and error files can now contain "%" character by specifying
    a file name with two consecutive "%" characters. For example,
    "sbatch -o "slurm.%%.%j" for job ID 123 will generate an output file named
    "slurm.%.123".
 -- Increase default sbcast buffer size from 512KB to 8MB.
 -- Add "ValidateTimeout" and "OtherTimeout" to "scontrol show burst" output.
 -- Implemented the checking configuration functionality using the new -C
    options of slurmctld. To check for configuration errors in slurm.conf
    run: 'slurmctld -C'.
 -- Burst buffer advanced reservation units treated as bytes (per documentation)
    rather than GB.
 -- Add "features_act" field (currently active features) to the node
    information. Output of scontrol, sinfo, and sview changed accordingly.
    The field previously displayed as "Features" is now "AvailableFeatures"
    while the new field is displayed as "ActiveFeatures".

OTHER CHANGES
=============
 -- Add mail wrapper script "smail" that will include job statistics in email
    notification messages.
 -- Removed support for authd. authd has not been developed and supported since
    several years.
 -- Enable the hdf5 profiling of the batch step.
 -- Eliminate redundant environment and script files for job arrays. This
    greatly reduces the number of files involved in managing job arrays.
 -- Burst buffer/cray - Add support for multiple buffer pools including support
    for different resource granularity by pool.
 -- Stop searching sbatch scripts for #PBS directives after 100 lines of
    non-comments. Stop parsing #PBS or #SLURM directives after 1024 characters
    into a line. Required for decent perforamnce with huge scripts.

API CHANGES
===========

Changed members of the following structs
========================================

Added members to the following struct definitions
=================================================

Added the following struct definitions
======================================

Changed the following enums and #defines
========================================

Added the following API's
=========================

Changed the following API's
============================

DBD API Changes
===============

Changed members of the following structs
========================================

Added the following struct definitions
======================================

Added members to the following struct definitions
=================================================

Added the following enums and #defines
========================================

Added the following API's
=========================

Changed the following API's
============================
