# Core Specialization

Core specialization is a feature designed to isolate system overhead (system interrupts, etc.) to designated cores on a compute node. This can reduce applications interrupts ranks to improve completion time. The job will be charged for all allocated cores, but will not be able to directly use the specialized cores.

## Command Options

All job allocation commands (salloc, sbatch and srun) accept the -S or --core-spec option with a core count value argument (e.g. "-S 1" or "--core-spec=2"). The count identifies the number of cores to be reserved for system overhead on each allocated compute node. Each job's core specialization count can be viewed using the scontrol, sview or squeue command. Use the squeue command with the "%X" format option to see the count (it is not reported in the default output format). The scontrol and sview commands can also be used to modify the count for pending jobs.

Configuring core specialization for a job implicitly sets its --exclusive option, reserving entire nodes for the job, however the number of CPUs available for the job's steps will not exceed that explicitly requested by the job. For example "salloc -n1 --core-spec=1 bash" will result in one entire node being reserved for the job, but only one CPU will be available for subsequent job steps. The remaining CPUs on the node (at least one core because of "--core-spec=1") will only be available for system use. The job will be changed for all CPUs on the node and the job's NumCPUs value reported by the scontrol, sview and squeue commands will reflect all CPUS on all allocated nodes as will the job's accounting.

## Core Selection

Cores selected for specialization will always follow the same assignment algorithm. The first core selected will be the highest numbered core on the highest numbered socket. Subsequent cores selected will be the highest numbered core on the lower numbered sockets. If additional cores are required, they well come from the next highest numbered cores on each socket. By way of example, consider a node with two sockets, each with four cores. The specialized cores will be selected in the following order:
socket: 1 core: 3
socket: 0 core: 3
socket: 1 core: 2
socket: 0 core: 2
socket: 1 core: 1
socket: 0 core: 1
socket: 1 core: 0
socket: 0 core: 0

Note that core specialization reservation may impact the use of some job allocation request options, especially --cores-per-socket.

## System Configuration

Core Specialization is only supported if SelectType is configured as either select/cons_res or select/cray. A job's core specialization option will be silently cleared on other configurations. In addition, each compute node's core count must be configured or the CPUs count must be configured to the node's core count. If the core count is not configured and the CPUs value is configured to the count of hyperthreads, then hyperthreads rather than cores will be reserved for system use.

Last modified 7 February 2014