#  $Id$
############################################################################### 
#                   Sample configuration file for SLURM
###############################################################################
#
# This file holds the system-wide SLURM configuration. It is read
# by SLURM clients, daemons, and the SLURM API to determine where
# and how to contact the SLURM controller, what other nodes reside
# in the current cluster, and various other configuration information.
#
# SLURM configuration parameters take the form Keyword=Value, where
# at this time, no spacing is allowed to surround the equals (=) sign.
# Many of the config values are not mandatory, and so may be left
# out of the config file. We will attempt to list the default 
# values for those parameters in this file.
#
###############################################################################

#  
#     SLURM daemon configuration
#     ========================================================================

#
# o Define the location of the SLURM controller and backup controller:
#    "ControlMachine"   : hostname of the primary controller
#    "ControlAddr"      : hostname used to contact the primary controller
#    "BackupController" : hostname of the backup controller 
#    "BackupAddr"       : hostname used to contact backup controller
#
# Example:
#
# ControlMachine=dev0
# ControlAddr=edev0		# default: same as ControlMachine
# BackupController=dev1		# default: no backup controller
# BackupAddr=edev1		# default: same as BackupController

#
# o Define the SLURM controller "save state" directory
#
#   The SLURM controller, slurmctld, will periodically save state
#   into this directory so that said state may be recovered after
#   a fatal system error. For best results when using a backup 
#   controller, the filesystem on which this directory resides 
#   should be shared between the "ControlMachine" and "BackupController"
#
# Example:
# 
# StateSaveLocation=/mnt/slurm	# default: "/tmp"


#
# o Define the slurmd "save state" directory
#
#   The SLURM daemon executing on each host, slurmd, will periodically 
#   save state into this directory so that said state may be recovered
#   after a fatal system error. This pathname is shared by all hosts, 
#   but the file must be unique on each host so this must reference a 
#   local file system.
#
# Example:
# 
# SlurmdSpoolDir=/var/tmp/slurmd	# default: "/tmp/slurmd"


#
# o Define the "slurm" user
#
#   "SlurmUser" specifies the user that the SLURM controller should run
#   as. The slurm controller has no need to run with elevated privileges,
#   so a user other than "root" is suggested here. 
#
# Example:
#
# SlurmUser=slurm


#
# o If you have a slow NIS environment,
#
#   big parallel jobs take a long time to start up (and may eventually
#   time-out) because the NIS server(s) may not be able to quickly
#   respond to simultaneous requests from multiple slurmd's.  You can
#   instruct slurmd to cache /etc/groups entries to prevent this from
#   happening by setting "CacheGroups=1".  Reconfiguring ("scontrol reconfig") 
#   with CacheGroups=0  will cause slurmd to purge the cache.
#
#   WARNING: The group ID cache does not try to keep itself in sync with
#            the system.  You MUST run "scontrol reconfig" to update the
#            cache after making any changes to system password or group
#            databases.
#
# Example:
#
# CacheGroups=1		# default is `0'


#
# o Define the slurmctld and slurmd server port numbers
#
#  by default, the slurmctld ports are set by checking for an entry in
#  /etc/services, and if that fails, by using an internal default set
#  at build time. That process will be overridden by these config 
#  parameters.
#
#    "SlurmctldPort"    : slurmctld server port 
#    "SlurmdPort"       : slurmd server port
#
# Example:
#
# SlurmctldPort=7010 	# 
# SlurmdPort=7011       #


#
# o Define slurmd and slurmctld logging options
#
#    "SlurmctldDebug"   : verbosity of slurmctld log messages 
#                         (Values from 0 to 7 are legal, with `0' being
#                          "quiet" operation and `7' being insanely verbose)
#
#    "SlurmdDebug"      : verbosity of slurmd log messages (0-7, see above)
#
#    "SlurmctldLogFile" : fully qualified pathname to slurmctld logfile
#                         (If a logfile is set for slurmctld, logging via
#                          syslog will be turned off)
#
#    "SlurmdLogFile"    : fully qualified pathname to slurmd logfile
#                         (same caveat as SlurmctldLogfile above)
#
# Example:
#
# SlurmctldDebug=4	# default is `3'  
# SlurmdDebug=4		# default is `3'
#
# SlurmctldLogFile=/var/log/slurmctld.log  # default is to log via syslog()
# SlurmdLogFile=/var/log/slurmd.log        # default is to log via syslog()


# o Define an alternate location for slurmd and slurmctld pid files, 
#   SlurmctldPidFile and SlurmdPidFile should have different values
#  
#    "SlurmctldPidFile" : fully qualified pathname containing slurmctld pid
#
#    "SlurmdPidFile"    : fully qualified pathname containing slurmd pid
#    
# Example:
#
# SlurmctldPidFile=/var/slurm/slurmctld.pid  # default: "/var/run/slurmctld.pid"
# SlurmdPidFile=/var/slurm/slurmd.pid     # default: "/var/run/slurmd.pid"

#
# o Define the authentication method for communicating between SLURM
#   components
#
# "auth/none"   : no authentication, the default
# "auth/authd"  : Brent Chun's authd
# "auth/munge"  : LLNL's munge
#
# WARNING: The use of "auth/none" permits any user to execute jobs as any 
# other user. This may be fine for testing purposes, but do not use it in production.
#
# AuthType=auth/none


#
# o Define a scheduler.
#
# "SchedulerType"	 : the type of scheduler. Orders pending jobs.
#	"sched/builtin"	 : the default, SLURM's built-in FIFO scheduler.
#	"sched/backfill" : FIFO scheduling with backfill.
#	"sched/hold"     : hold all new jobs if "/etc/slurm.hold" exists, 
#	                   otherwise perform FIFO scheduling.
#	"sched/wiki"	 : the Wiki interface to Maui.
#
# "SchedulerAuth"	 : an authentication token, if any, that must
#			   be used in a scheduler communication
#			   protocol.  The interpretation of this value
#			   depends on the plugin type.
#
# "SchedulerPort"	 : for polling schedulers, the port number on
#			   which slurmctld should listen for connection
#			   requests.
#
# "SchedulerRootFilter"	 : for schedulers that support it (currently only
#			   sched/backfill). If set to '1' then scheduler
#			   will filter and avoid RootOnly partitions (let
#			   root user or process schedule these partitions).
#			   Otherwise scheduler will treat RootOnly
#			   partitions as any other standard partition.
#
# SchedulerType=sched/wiki
# SchedulerAuth=42
# SchedulerPort=7321
# SchedulerRootFilter=0	# default is '1'


#
# "SelectType"			: node selection logic for scheduling.
#	"select/bluegene"	: the default on BlueGene systems, aware of
#				  system topology, manages bglblocks, etc.
#	"select/cons_res"	: allocate individual consumable resources
#				  (i.e. processors, memory, etc.)
#	"select/linear"		: the default on non-BlueGene systems,
#				  no topology awareness, oriented toward
#				  allocating nodes to jobs rather than
#				  resources within a node (e.g. CPUs)
#
# SelectType=select/linear


#
# "JobCompType"			:  Define the job completion logging
#				   mechanism to be used
#	"jobcomp/none"		: no job logging, the default
#	"jobcomp/filetxt"	: log job record to a text file
#	"jobcomp/script"	: execute an arbitrary script
#
# JobCompType=jobcomp/filetxt


#
# o Define location where job completion logs are to be written
#   Interpretation of the parameter is dependent upon the logging
#   mechanism used (specified by the JobCompType parameter). For
#   "JobCompType=jobcomp/filetxt" the value of JobCompLoc should
#   be the fully qualified pathname of a file into which text
#   records are appended.
#
# JobCompLoc=/var/log/slurm.job.log


#
# o Define the switch or interconnect in use.
#
# "SwitchType"         : the type of switch or interconnect.
#     switch/none      : the default, supports all switches not requiring
#                        special set-up for job launch including Myrinet, 
#                        Ethernet, and InfiniBand.
#     switch/federation: IBM Federation switch
#     switch/elan      : Quadrics Elan 3 or Elan 4 interconnect.
#
# SwitchType=switch/none


#
# o Define the process tracking mechanism in use.
#
# "ProctrackType"           : the type of process tracking mechanism
#     "proctrack/aix"       : use AIX kernel extension for process tracking,
#                             the default value on AIX computers
#     "proctrack/pgid"      : use Unix process group ID for process tracking,
#                             the default value on all other computers
#     "proctrack/linuxproc" : use parent process ID to establish process
#                             tree, required for MPICH-GM use
#     "proctrack/rms"       : use Quadrics kernal infrastructure to track 
#                             processes, strongly recommended for systems
#                             with a Quadrics switch
#
# ProctrackType=proctrack/pgid


#
# o Define the job accounting mechanism to use.
#
#   "jobacct/log"    : Generate job accounting data for each job.
#   "jobacct/none"   : Do not track accounting data.
#

JobAcctType=jobacct/none

#
# o Define the location where job accounting logs are to be
#   written. For
#    - jobacct/none - this parameter is ignored
#    - jobacct/log  - the fully-qualified file name for the data file
#
# JobAcctLoc=/var/log/slurm_accounting.log

#
# o Define any parameters to pass to the job accounting plugin. For
#    - jobacct/none - this parameter is ignored
#    - jobacct/log  - the parameters available are
#          - Frequency=N       - where N is the number of seconds
#                                between sampling the memory usage stats
#                                (psize, vsize). If N=0, no sampling is
#                                done, and psize and vsize are returned
#                                as 0.
#          - MaxSendRetries    - Number of times to try to send an
#                                accounting message before giving up.
#          - MaxSendRetryDelay - Pause for 1 to MaxSendRetryDelay
#                                seconds between attempts to deliver an
#                                accounting message.
#          - StaggerSlotSize   - For a process that might be sending
#                                a message at the same time as N other
#                                processes in the job, where N is 10 or greater,
#                                each process will pause a bit before
#                                trying to send its message. For N tasks,
#                                N "staggered timeslots" are defined,
#                                in increments of (StaggerSlotSize*.001)
#                                seconds. The first process sends its
#                                message immediately, the second process
#                                pauses one increment before sending,
#                                the third process pauses two increments
#                                before sending, and so on.  
#
# JobAcctParameters="Frequency=30,MaxSendRetries=5,MaxSendRetryDelay=120,\
#                     StaggerSlotSize=5" 
#
# o Define the places to look for SLURM plugins.  This is a
#   colon-separated list of directories, just like the PATH
#   environment variable.
#
# PluginDir=/etc/slurm/plugins # default: PREFIX/lib/slurm


#
# o Define some timeout values for the slurm controller and backup
#
#    "SlurmctldTimeout" : amount of time, in seconds, backup controller
#                         waits for primary controller to respond 
#                         before assuming control.
#
#    "SlurmdTimeout"    : amount of time, in seconds, the controller
#                         waits for slurmd to respond before setting the
#                         node's state to DOWN. If set to 0, this feature
#                         is disabled.
#
#    "HeartbeatInterval": The interval, in seconds, at which the SLURM
#                         controller tests the status of other daemons.
#
#    "InactiveLimit"    : The interval, in seconds, a job or job step is 
#                         permitted to be inactive (srun command not responding)
#                         before being terminated.
#
#    "MinJobAge"        : The time, in seconds, after a job completes before
#                         its record is purged from the active slurmctld data.
#   
#    "KillWait"         : The time, in seconds, between SIGTERM and SIGKILL
#                         signals sent to a job upon reaching its timelimit.
#
#    "WaitTime"         : Specifies how many seconds srun should wait after the 
#                         first task terminates before terminating all remaining  
#                         tasks. If set to 0, this feature is disabled.
#
# Example:
#
# SlurmctldTimeout=300	# Defaults to 300 seconds
# SlurmdTimeout=300	# Defaults to 300 seconds
# HeartBeatInterval=30	# Defaults to 30 seconds
# InactiveLimit=600	# Defaults to 0 (unlimited)
# MinJobAge=30		# Defaults to 300 seconds
# KillWait=10		# Defaults to 30 seconds
# WaitTime=30		# Defaults to 0 (unlimited)


#
# o Define other miscellaneous SLURM controller configuration values:
#
#    "FastSchedule"     : if set to `1' consider the configuration of nodes
#                         to be exactly that set in the config file. Otherwise,
#                         consider configuration of nodes to that which is
#                         reported by the node's slurmd. A FastSchedule value of
#                         zero will result in significantly slower scheduling.
#
#    "FirstJobId"       : Number of the first assigned job id.
#
#    "ReturnToService"  : if set to `1,' nodes in the DOWN state will be
#                         set to IDLE after they come back up. Otherwise,
#                         nodes will stay in the down state until manually
#                         brought into the IDLE state.
# 
#    "MaxJobCount"      : defines the maximum number of jobs slurmctld can 
#                         have in its active database at one time. Set the 
#                         values of MaxJobCount and MinJobAge so as to avoid 
#                         having slurmctld exhaust its memory or other resources.
#
#    "MpiDefault"	: define the default type of MPI to be used. If
#			  srun does not specify another value, slurm will 
#			  establish the environment for this mpi to execute.
#			  Currently supported values are lam (for LAM MPI and 
#                         Open MPI), mpich-gm, mvapich, and none (default, 
#                         which works for most other versions of MPI).
#
# Example:
#
# FastSchedule=0		# default is `1'
# FirstJobid=1000       	# default is `1'
# ReturnToService=1     	# default is `0'
# MaxJobCount=10000		# Defaults to 2000
# MpiDefault			# default is "none"


#
# o Define the Resource Limit Propagation Configuration
#
#   These two parameters can be used to specify which resource limits to
#   propagate from the users environment on the submit node to the users job
#   environment on the compute nodes.  This can be useful when system limits
#   vary among nodes.  By default, (when neither parameter is  specified), all
#   resource limits are propagated.   The values of non-propagated resource
#   limits are determined by the system limits configured on the compute
#   nodes.   Only one of these two parameters may be specified.
#
#    "PropagateResourceLimits"       : A list of one or more comma-separated
#                                      resource limits whose (soft) values
#                                      will be set at job startup on behalf of
#                                      the user.  Any resource limit that is
#                                      not listed here, will not be propagated,
#                                      (unless the user overrides this setting
#                                      with the 'srun --propagate' switch).
#
#
#    "PropagateResourceLimitsExcept" : A list of one or more comma-separated
#                                      resource limits which will not be
#                                      propagated.  Any resource limit that is
#                                      not listed here, will be propagated.
#   
#                                The following resource limits are supported:
#
#                                RLIMIT_NPROC   RLIMIT_MEMLOCK   RLIMIT_CORE
#                                RLIMIT_FSIZE   RLIMIT_CPU       RLIMIT_DATA
#                                RLIMIT_STACK   RLIMIT_RSS       RLIMIT_NOFILE
#                                RLIMIT_AS
#
# Examples:
#
# PropagateResourceLimits=RLIMIT_CORE,RLIMIT_DATA # The users RLIMIT_CORE and
#                                                 # RLIMIT_DATA resource limit
#                                                 # soft values will be applied
#                                                 # to the job on startup.  All
#                                                 # other resource limit soft
#                                                 # values are determined by the
#                                                 # system limits defined on
#                                                 # the compute nodes.
#
# PropagateResourceLimitsExcept=RLIMIT_MEMLOCK    # All limits, except for
#                                                 # MEMLOCK, are propagated.
#



#
# o Define an epilog and a prolog
#
#    "Prolog" : fully qualified path to script that will be executed as 
#               root on every node of a user's job before the job's tasks
#               will be initiated there.
#
#    "Epilog" : fully qualified path to a script that will be executed as
#               root on every node of a user's job after that job has 
#               terminated.
#
# Example:
#
# Prolog=/usr/local/slurm/prolog	# default is no prolog
# Epilog=/usr/local/slurm/epilog	# default is no epilog


# 
# o Define programs to be executed by srun at job step initiation and 
#   termination. These parameters may be overridden by srun's --prolog 
#   and --epilog options.
#
# Example:
#
# SrunProlog=/usr/local/slurm/srun_prolog   # default is no srun prolog
# SrunEpilog=/usr/local/slurm/srun_epilog   # default is no srun epilog


#
# o Define task launch specific parameters
#
#    "TaskProlog" : Define a program to be executed as the user before each 
#                   task begins execution.
#    "TaskEpilog" : Define a program to be executed as the user after each 
#                   task terminates.
#    "TaskPlugin" : Define a task launch plugin. This may be used to 
#                   provide resource management within a node (e.g. pinning
#                   tasks to specific processors). Permissible values are:
#      "task/none"     : no task launch actions, the default.
#      "task/affinity" : CPU affinity support (see "srun --cpu_bind=")
#
# Example:
#
# TaskProlog=/usr/local/slurm/etc/task_prolog # default is none
# TaskEpilog=/usr/local/slurm/etc/task_epilog # default is none
# TaskPlugin=task/affinity                    # default is task/none


#
# o Define the temporary file system 
#
#    "TmpFS"  : Defines the location of local temporary storage filesystem 
#               on remote nodes. This filesystem will be used in reporting
#               each node's TmpDisk space.
#
# Example:
#
# TmpFs=/var/tmp	# default "/tmp"


#
# o Define the location of the private and public keys used by SLURM
#   to generate job credentials.
#
#    "JobCredentialPrivateKey"       : Full pathname to the private key
#
#    "JobCredentialPublicCertificate : Full pathname to the public cert.
#
# Example:
#
# JobCredentialPrivateKey=/etc/slurm/slurm.key
# JobCredentialPublicCertificate=/etc/slurm/slurm.cert 


#
#     Node and Partition Configuration
#     ========================================================================

#
#  o Node configuration
#
#    The configuration information of nodes (or machines) to be managed 
#    by SLURM is described here. The only required value in this section
#    of the config file is the "NodeName" field, which specifies the 
#    hostnames of the node or nodes to manage. It is recommended, however,
#    that baseline values for the node configuration be established
#    using the following parameters (see slurm.config(5) for more info): 
#
#     "NodeName"   : The only required node configuration parameter, NodeName
#                    specifies a node or set of nodes to be managed by SLURM.
#                    The special NodeName of "DEFAULT" may be used to establish
#                    default node configuration parameters for subsequent node
#                    records. Typically this would be the string that 
#                    `/bin/hostname -s` would return on the node. However 
#                    NodeName may be an arbitrary string if NodeHostname is 
#                    used (see below).
#
#     "Feature"    : comma separated list of "features" for the given node(s) 
#
#     "NodeAddr"   : preferred address for contacting the node. This may be 
#                    either a name or IP address.
#
#     "NodeHostname"
#                  : the string that `/bin/hostname -s` would return on the
#                    node.  In other words, NodeName may be the name other than
#                    the real hostname.
#
#     "RealMemory" : Amount of real memory (in Megabytes)
#
#     "Procs"      : Number of CPUs 
#
#     "State"      : Initial state (IDLE, DOWN, etc.)
#
#     "TmpDisk"    : Temporary disk space available on node
#
#     "Weight"     : Priority of node for scheduling purposes
#
#   If any of the above values are set for a node or group of nodes, and
#   that node checks in to the slurm controller with less than the 
#   configured resources, the node's state will be set to DOWN, in order
#   to avoid scheduling any jobs on a possibly misconfigured machine.
#
# Example Node configuration:
#
# NodeName=DEFAULT Procs=2 TmpDisk=64000 State=UNKNOWN
# NodeName=host[0-25] NodeAddr=ehost[0-25] Weight=16
# NodeName=host26     NodeAddr=ehost26     Weight=32 Feature=graphics_card

#
# o Partition Configuration
#
#   Paritions are groups of nodes which (possibly) have different limits
#   and access controls. Nodes may be in multiple partitions. Jobs will
#   not be allowed to span partitions. The following partition configuration
#   parameters are recognized:
#
#    "PartitionName" : Name used to reference this partition. The special
#                      PartitionName of "DEFAULT" may be used to establish
#                      default partition configurations parameters for 
#                      subsequent partition records.
#
#    "Nodes"         : list of nodes that compose this partition
#
#    "AllowGroups"   : Comma separated list of group ids which are allowed
#                      to use the partition. Default is "ALL" which allows
#                      all users to access the partition.
#
#    "Default"       : if "YES" the corresponding partition will be the 
#                      default when users submit jobs without specification
#                      of a desired partition.
#
#    "RootOnly"      : only user id zero (root) may use this partition
#
#    "MaxNodes"      : Maximum count of nodes that will be allocated to any
#                      single job. The default is unlimited or `-1'
#
#    "MaxTime"       : Maximum timelimit of jobs in this partition in minutes.
#                      The default is unlimited or `-1'
#
#    "MinNodes"      : Minimum count of nodes that will be allocated to any
#                      single job. The default is `1'
#
#    "Shared"        : Allow sharing of nodes by jobs. Possible values are
#                      "YES" "NO" or "FORCE"
#
#    "State"         : State of partition. Possible values are "UP" or "DOWN"
#
#
# Example Partition Configurations:
#
# PartitionName=DEFAULT MaxTime=30 MaxNodes=26
# PartitionName=debug Nodes=host[0-8,18-25] State=UP Default=YES
# PartitionName=batch Nodes=host[9-17,26]   State=UP
#
#

#
