bug 1379

bug 1375

based upon email from Taras to slurm mailing list

What this does is use the core level binding after each task is laid out
to skip all the extra threads in the core so it doesn't give them to
another task.

It probably isn't perfect, but does solve all the scenarios I found.

Fix for GRES scheduling in which there is CPU topology defined or
GRES types defined and there is more than 1 GPU per topology record
in slurmctld. Without this fix, only one GRES could be allocated
from each defined topology.
bug 1369

The slurmctld could abort with a gres configuration having
Type= configured, but no CPU binding configured.

Revert a changed definition used only in the debugging work from commit
ac9436bc

Make sure correct locks are held when messing with the assoc_mgr cache.

Most of these don't matter as they are all NO_LOCK

Fallout from commit f1ebdef1 when the resources were added.

This is needed for setups like this

TaskPlugin = affinity
TaskPlugin = task/affinity,task/cgroup
TaskPlugin = affinity,cgroup

Slurm 14.03 nccs

Intel MPI, on MPI jobs initialisation through PMI, uses to call PMI_KVS_Put()
many many times from task at rank 0, and each on these call is followed by
PMI_KVS_Commit(). Slurm implementation of PMI_KVS_Commit() imposes a delay
to avoid DDOS on original srun. This delay is proportional to the total number.
It could be up to 3 secs for large jobs for ex. with 7168 tasks. Therefore,
when Intel MPI calls PMI_KVS_Commit() 475 times (mesured on a test case) from
task at rank 0, 28 minutes are spent in delay function.
All other tasks in the job are waiting for a PMI_Barrier. Therefore, there is
no risk for a DDOS from this single task 0. The patch alters the delaying time
calculation to make sure task at rank 0 will does not be delayed. All other
tasks are globally spreaded in the same time range as before.

Ammendment to commit 744f114b

Fix race condition that could start a job that is dependent upon a job array
before all tasks of that job array complete.
bug 1324

Conflicts:
	src/sbatch/opt.c

Bug 1350

flag from a job while the job is waiting for a block to boot.

because of the referenced commit.  ntasks_set is always true on a BGQ at
this point.