SPANK(5)

NAME
SPANK - SLURM Plug-in Architecture for Node and job (K)control

DESCRIPTION
This manual briefly describes the capabilities of the SLURM Plug-in architecture for Node and job Kontrol (SPANK) as well as the SPANK configuration file: (By default: plugstack.conf.)

SPANK provides a very generic interface for stackable plug-ins which may be used to dynamically modify the job launch code in SLURM. SPANK plugins may be built without access to SLURM source code. They need only be compiled against SLURM's spank.h header file, added to the SPANK config file plugstack.conf, and they will be loaded at runtime during the next job launch. Thus, the SPANK infrastructure provides administrators and other developers a low cost, low effort ability to dynamically modify the runtime behavior of SLURM job launch.

SPANK plugins may be called from multiple points during the SLURM job launch. A plugin may define the following functions:

slurm_spank_init
    Called just after job step is initialized.

slurm_spank_user_init
    Called after privileges are temporarily dropped.

slurm_spank_task_init
    Called for each task just before execve(2).

slurm_spank_task_post_fork
    Called for each task from parent process after fork(2) is complete. Due to the fact that slurmd does not exec any tasks until all tasks have completed fork(2), this call is guaranteed to run before the user task is executed.

slurm_spank_task_exit
    Called for each task as its exit status is collected by SLURM.

slurm_spank_exit
    Called once just before slurmstepd exits.

All of these functions have the same prototype, for example:
   slurm_spank_init (spank_t spank, int ac, char *argv[])

Where spank is the SPANK handle which must be passed back to SLURM when the plugin calls functions like spank_get_item and spank_getenv. Configured arguments (See CONFIGURATION below) are passed in the argument vector argv with argument count ac.

SPANK plugins do not have direct access to internally definied SLURM data structures. Instead, information about the currently executing job is obtained via the spank_get_item function call.
  spank_err_t spank_get_item (spank_t spank, spank_item_t item, ...);
The spank_get_item call must be passed the current SPANK handle as well as the item requested, which is defined by the passed spank_item_t. A variable number of pointer arguments are also passed, depending on which item was requested by the plugin. A list of the valid values for item is kept in the spank.h header file. Some examples are:

S_JOB_UID
    User id for running job. (uid_t *) is third arg of spank_get_item

S_JOB_STEPID
    Job step id for running job. (uint32_t *) is third arg of spank_get_item

S_TASK_EXIT_STATUS
    Exit status for exited task. Only valid from slurm_spank_task_exit. (int *) is third arg of spank_get_item.

S_JOB_ARGV
    Complete job command line. Third and fourth args to spank_get_item are (int *, char ***).

See spank.h for more details, and EXAMPLES below for an example of spank_get_item usage.

SPANK plugins may also use the spank_getenv and spank_setenv functions to view and modify the job's environment. spank_getenv searches the job's environment for the environment variable var and copies the current value into a buffer buf of length len. spank_setenv allows a SPANK plugin to set or overwrite a variable in the job's environment. The prototypes are:
 spank_err_t spank_getenv (spank_t spank, const char *var,
		           char *buf, int len);
 spank_err_t spank_setenv (spank_t spank, const char *var,
		           const char *val, int overwrite);

See spank.h for more information, and EXAMPLES below for an example for spank_getenv usage.

CONFIGURATION

The default SPANK plug-in stack configuration file is plugstack.conf in the same directory as slurm.conf(5), though this may be changed via the SLURM config parameter PlugStackConfig. The config file lists SPANK plugins, one per line, along with whether the plugin is required or optional, and any global arguments that are to be passed to the plugin for runtime configuration. Comments are preceded with '#' and extend to the end of the line. If the configuration file is missing or empty, it will simply be ignored.

The format of each non-comment line in the configuration file is:
  required/optional   plugin   arguments
For example:
  optional /usr/lib/slurm/test.so
Tells slurmd to load the plugin test.so passing no arguments. If a SPANK plugin is required, then failure of any of the plugin's functions will cause slurmd to terminate the job, while optional plugins only cause a warning.

SPANK plugins are stackable, meaning that more than one plugin may be placed into the config file. The plugins will simply be called in order, one after the other, and appropriate action taken on failure given that state of the plugin's optional flag.

The SPANK config file is re-read on each job launch, so editing the config file will not affect running jobs. However care should be taken so that a partially edited config file is not read by a launching job.

EXAMPLES

Simple SPANK config file:
#
# SPANK config file
#
# required?       plugin                     args
#
optional          /usr/lib/slurm/renice.so   verbose
required          /usr/lib/slurm/test.so

Simple SPANK plugin to modify job nice value. Allows priority to be set via SLURM_RENICE environment variable, though system-wide default could be set using SPANK plugin arguments in the configuration file.
#include <sys/types.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/resource.h>
#include <slurm/spank.h>
SPANK_PLUGIN(renice, 1);
int slurm_spank_task_post_fork (spank_t sp, int ac, char **av)
{
        pid_t pid;
        int taskid;
        char buf [1024];
        long int prio;
        const char *var = "SLURM_RENICE";
        char *p;
        spank_get_item (sp, S_TASK_GLOBAL_ID, &taskid);
        spank_get_item (sp, S_TASK_PID, &pid);
        if (spank_getenv (sp, var, buf, 1024) != ESPANK_SUCCESS) {
                return (0);
        }
        prio = strtol (buf, &p, 10);
        if ((*p != ' ') || (prio < -20) || (prio > 20)) {
                slurm_error ("Bad value for %s:
                return (-1);
        }
        slurm_info ("re-nicing task%d pid %ld to %ld,
                    taskid, pid, prio);
        if (setpriority (PRIO_PROCESS, (int) pid, (int) prio) < 0) {
                slurm_error ("setpriority: %m");
                return (-1);
        }
        return (0);
}

COPYING
Copyright (C) 2002-2006 The Regents of the University of California. Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). UCRL-CODE-217948.

This file is part of SLURM, a resource management program. For details, see <http://www.llnl.gov/linux/slurm/>.

SLURM is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

SLURM is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

FILES
/etc/slurm/slurm.conf - SLURM configuration file.
/etc/slurm/plugstack.conf - SPANK configuration file.
/usr/include/slurm/spank.h - SPANK header file.

SEE ALSO

slurm.conf(5)