Cray and BlueGene - Do not treat lack of usable front-end nodes when
slurmctld deamon starts as a fatal error. Also preserve correct front-end
node for jobs when there is more than one front-end node and the slurmctld
daemon restarts.

while it is attempting to free underlying hardware is marked in error
making small blocks overlapping with the freeing block.  This only
applies to dynamic layout mode.

This corresponds to commit dd2dce54
from Mark Grondona's work in squeue, but applied to the sview command.

Slurm 2.4 minor fixes

This addresses trouble ticket 85

runjob_mux

is always set when sending or receiving a message.

Set SLURM_CONF in default prolog/epilog environment instead
of only in spank prolog/epilog environment.

This change fixes a potential hang during spank prolog/epilog
execution due to the possibility of memory allocation after
fork(2) and before exec(2) when invoking slurmstepd spank
prolog|epilog.

This also has the benefit that SLURM commands used in prolog and epilog
scripts will use the correct slurm.conf file.

If exec_wait_child_wait_for_parent() fails for any reason, it is safer
to abort immediately rather than proceed to execute the user's job.

On a failure of fork(2), slurmstepd would print an error and exit,
possibly leaving previously forked children waiting.

Ensure a better cleanup by killing all active children on fork failure
before exiting slurmstepd.

Close the read end of the pipe slurmstepd uses to notify children
it is time to call exec(2) in order to save one file descriptor per
task. (Previously, the read side of the pipe wasn't closed until
exec_wait_info was destroyed)

For some reason squeue was treating completing jobs the same as
pending jobs, and reported the number of nodes as the maximum of
requested nodelist, requested node count or CPUs (divided into nodes?)

This is in contrast to the squeue manpage which explicitly states
that the number of nodes reported for completing jobs should
be only the nodes that are still allocated to the job.

This patch removes the special handling of completing jobs in
src/squeue/print.c:_get_node_cnt(), so that the squeue output for
completing jobs matches documentation. A comment is also added
so that developers looking at the code understand what is going on.

than 1 midplane but not the entire allocation.

multi midplane block allocation.

where other blocks on an overlapping midplane are running jobs.

hardware is marked bad remove the larger block and create a block over
just the bad hardware making the other hardware available to run on.

allocation.

for a job to finish on it the number of unused cpus wasn't updated
correctly.

See Bugzilla #73 for more complete description of the problem.
Patch by Martin Perry, Bull.

If job is submitted to more than one partition, it's partition pointer can
be set to an invalid value. This can result in the count of CPUs allocated
on a node being bad, resulting in over- or under-allocation of its CPUs.
Patch by Carles Fenoy, BSC.

Hi all,

After a tough day I've finally found the problem and a solution for 2.4.1
I was able to reproduce the explained behavior by submitting jobs to 2 partitions.
This makes the job to be allocated in one partition but in the schedule function the partition of the job is changed to the NON allocated one. This makes that the resources can not be free at the end of the job.

I've solved this by changing the IS_PENDING test some lines above in the schedule function in (job_scheduler.c)

This is the code from the git HEAD (Line 801). As this file has changed a lot from 2.4.x I have not done a patch but I'm commenting the solution here.
I've moved the if(!IS_JOB_PENDING) after the 2nd line (part_ptr...). This prevents the partition of the job to be changed if it is already starting in another partition.

job_ptr = job_queue_rec->job_ptr;

part_ptr = job_queue_rec->part_ptr;
job_ptr->part_ptr = part_ptr;
xfree(job_queue_rec);

if (!IS_JOB_PENDING(job_ptr))

continue; /* started in other partition */

Hope this is enough information to solve it.

I've just realized (while writing this mail) that my solution has a memory leak as job_queue_rec is not freed.

Regards,
Carles Fenoy