Since we don't currently track energy usage per task (only per step).
Otherwise we get double the energy.

Andy Wettstein (University of Chicago) reported privately to me that slurmctld
2.5.4 crashed after he enabled the priority/multifactor2 plugin due to a
division by zero error.

I was able to reproduce the crash by creating an account hierarchy where all
the accounts and users had zero shares.
See bug 315

Revert premature change of META

Without this change, it appears that POE ignores the -procs argument
resulting in a job step request with multiple host names, but only
one ntask required

For example "host1*2" is equivalent to "host1,host1".

Without this change, it appears that POE ignores the -procs argument
resulting in a job step request with multiple host names, but only
one ntask required

We're having some trouble getting our slurm jobs to successfully
restart after a checkpoint.  For this test, I'm using sbatch and a
simple, single-threaded executable.  Slurm is 2.5.4, blcr is 0.8.5.
I'm submitting the job using sbatch:

$ sbatch -n 1 -t 12:00:00 bin/bowtie-ex.sh

I am able to create the checkpoint and vacate the node:

$ scontrol checkpoint create 137
.... time passes ....
$ scontrol vacate 137

At that point, I see the checkpoint file from blcr in the current
directory and the checkpoint file from Slurm
in /var/spool/slurm-llnl/checkpoint.  However, when I attempt to
restart the job:

$ scontrol checkpoint restart 137
scontrol_checkpoint error: Node count specification invalid

In slurmctld's log (at level 7) I see:

[2013-05-29T12:41:08-07:00] debug2: Processing RPC: REQUEST_CHECKPOINT(restart) from uid=*****
[2013-05-29T12:41:08-07:00] debug3: Version string in job_ckpt header is JOB_CKPT_002
[2013-05-29T12:41:08-07:00] _job_create: max_nodes == 0
[2013-05-29T12:41:08-07:00] _slurm_rpc_checkpoint restart 137: Node count specification invalid

Rename slurm_step_ctx_params_t field from "mem_per_cpu" to "pn_min_memory".
Job step now accepts memory specification in either per-cpu or per-node basis.

Uninitialized variables resulted in error of
"cons_res: sync loop not progressing, holding job #"

The most notable problem case is on a cray where a job step
specifically requests one or more node that are not the first
nodes in the job allocation

If set, the job's size component will be based upon not the job size
alone, but the job's size divided by it's time limit.

The problem we have observed is the backfill scheduler temporarily
gives up its locks (one second), but then reclaims them before the
backlog of work completes, basically keeping the backfill scheduler
running for a really long time when under a heavy load.
bug 297

Defers (rather than forgets) reboot request with job running on the
node within a reservation.

Defers (rather than forgets) reboot request with job running on the
node within a reservation.

Previously nodes in a MAINT reservation would NOT reboot on demand.

all preemptable jobs on the midplane instead of just the ones it needed to.

This bug was introduced in commit f1cf6d2d
fix for bug 290

Downing the node will kill all jobs allocated to the node, very bad
on something like a BlueGene system

This can be especially useful to schedule GPUs. For example a node can be
associated with two Slurm partitions (e.g. "cpu" and "gpu") and the
partition/queue "cpu" could be limited to only a subset of the node's CPUs,
insuring that one or more CPUs would be available to jobs in the "gpu"
partition/queue.

the node tab and we didn't notice.

The option "interval" was the original parameter, but has not been
documented for a year or more.

Without this change pmdv12 was bound to one CPU and could not use
all of the resources allocated to the job step for the tasks that
it launches

This only changes behaviour when the --ntasks option is not used,
but the --cpus-per-task option is use

Also add size specification of "%0" to not limit a field size.
For example "sacct --format=%0ALL" to print everything.