Decrease maximimum scheduler main loop run time from 10 secs to
4 secs for improved performance.
If running with sched/backfill, do not run through all jobs on
periodic scheduling loop, but only the default depth. The
backfill scheduler can go through more jobs anyway due to its
ability to relinquish and recover locks.
See bug 616

if an job step's network value is set by poe, either by directly
executing poe or srun launching poe, that value was not being
propagated to the job step creation RPC and the network was not
being set up for the proper protocol (e.g. mpi, lapi, pami, etc.).
The previous logic would only work if the srun execute line
explicitly set the protocol using the --network option.

Do not overcommit partitions with PreemptMode=off

Prevent preemption of jobs in partition where PreemptMode=off

processes.

When slurmctld restarted, it would not recover dependencies on
job array elements and would just discard the depenency. This
corrects the parsing problem to recover the dependency. The old code
would print a mesage like this and discard it:
slurmctld: error: Invalid dependencies discarded for job 51: afterany:47_*

be setup for 1 node jobs.  Here are some of the reasons from IBM...

1. PE expects it.
2. For failover, if there was some challenge or difficulty with the
   shared-memory method of data transfer, the protocol stack might
   want to go through the adapter instead.
3. For flexibility, the protocol stack might want to be able to transfer
   data using some variable combination of shared memory and adapter-based
   communication, and
4. Possibly most important, for overall performance, it might be that
   bandwidth or efficiency (BW per CPU cycles) might be better using the
   adapter resources.  (An obvious case is for large messages, it might
   require a lot fewer CPU cycles to program the DMA engines on the
   adapter to move data between tasks, rather than depend on the CPU
   to move the data with loads and stores, or page re-mapping -- and
   a DMA engine might actually move the data more quickly, if it's well
   integrated with the memory system, as it is in the P775 case.)

than you really have.

doesn't get chopped off.

    a minus sign for options was intended.

Some of these were resulting in the state of a job not being updated
correctly to tools like sview.

in waiting reason ReqNodeNotAvail.

Add support for job array options in the qsub command, in #PBS
options for sbatch scripts and set the appropriate environment
variables in the spank_pbs plugin (PBS_ARRAY_ID and PBS_ARRAY_INDEX).
Note that Torque uses the "-t" option and PBS Pro uses the "-J"
option.

slurm.conf.  Rebooting daemons after adding nodes to the slurm.conf
is highly recommended.

Rather than continuously retrying a step create for suspended jobs,
add a sleep with exponential backoff

If a job is suspended, log the step create failure using debug
rather than info in slurmctld

The test for NRT_NULL_MAGIC failed to capture some problems
if the pointer to the structure was NULL. This is an ammendment
to commit 2a55aa0b

Perhaps should also look into doing this for nodeinfo and libstate

this would cause pmd's to hang.