Continuation of d0deea4f.

Bug 4841.

Use setenv() instead of setenvfs(), since setenvfs() memory allocation
is implemented with xmalloc() and fini_setproctitle() (which is called
on reconfigure) free's the memory with free(), leading to a:
"free(): invalid size" malloc_printerr error.

Continuation of dce83a23.

Bug 5095.

Due to current design the job limits are checked before the allocation is
made when one specifies a generic gres and a specific gres type is configured.
The workaround for now is to define a job submit plugin to control the user
request and succesfully apply limits.

Bug 4767

Bug 4960.

Bug 4887.

Can lead to deadlock within malloc depending on the exact timing.

Rework thread startup and shutdown code so pthread_cancel is not
needed.

Bug 5119, 5103.

happens.

Bug 5108

This reverts commit de5a4da2.

happens.

Bug 5108

Turns out the partititon's billing tres was working off the sum of
the node_ptrs which contain the max of all partitions they are in.

This isn't correct since each partition's billing can be different.

Set it correctly here.

No functional change.

The use in _watch_tasks needs to be removed as the switch to pthread_signal
from pthread_cancel means this will not get interrupted and would keep the
step alive for at least a second, potentially harming throughput.
Since the call to _poll_data() happens after the first timer expires,
this delay turns out to be unnecessary, so we won't be replacing it with
a pthread_cond_timedwait() construct.

The use jobacct_gather_stat_task() is unnecessary since the two locations
this can happen take place after _fork_all_tasks() has setup the tasks,
thus the delay should not be necessary.

Bug 5103.

These functions are not async-cancel-safe, and cannot safely be cancelled.
This leads to potential deadlock, either between our own locks, or deep
inside glibc when the thread held a malloc arena lock when canceled.

Replace with pthread_signal to the appropriate cond to
wake threads up at the appropriate time instead.

Bug 5103.

This will make it easier in a future commit to avoid the
async pthread_cancel.

Bug 5103

Bug 5110.

Remove partition MaxTime limit at the beginning of the test,
run the rest of the test, then restore the partition configuration
with scontrol reconfigure.

Bug 4994.

Otherwise the extern step will disappear after 11.5 days.

Bug 5000.

to be sure if it is created under job write lock.

Bug 4901

In conjuction with previous commit (reconginizing nodes being powered up
out of band) set node's last_idle to 0 when the node is in a power_save
state. Additional meaning that the node isn't booted.

Partially reverts da722a89. Checking for (last_idle > 0) when in
power_save state isn't necessary because if the node is already in
power_save state the node won't be resumed unless
(node_ptr->last_idle > (now - SuspendTime)). And with the previous
change, the node's last_idle time will be set when the node registers.

Bug 5053

This allows the suspend script to be triggered even if Slurm has the
node(s) in a power_save state.

Bug 5053

The test was failing solidly on a Cray with NHC configured

Disable the tests as needed

by hwloc_obj_type_snprintf.

You will only see this if you have _DEBUG set to 1.

The included lightweight corefile description is no longer valid,
but misleading at best.

When any of these --exclusive modes couldn't be satisfied, Slurm was
returning an incorrect ESLURM_NODE_NOT_AVAIL, having as a consequence
scheduling problems as described in the bug. The fix makes it so the
error code is properly set to ESLURM_NODES_BUSY, fixing also the
scheduling problems and working over the correct share_node_bitmap.

Continuation of commits from bug 4932:
e2a14b8d
fc4e5ac9

Bug 5047.

Fix an issue in the bit manipulation log introduced in commit 892ffa89.

Bug 4997.

And related KillOnBadExit setting in slurm.conf.

These only affect an individual job step, not the entire job.

Bug 5023.

Replace select_p_select_jobinfo_sprint() with the same NO-OP
that the other plugins (except alps and bluegene) have implemented.

Bug 5077.

Bug 5049

1. Identifies nodes which are unavailable to a specific job,
adding a call to filter_by_node_owner() in select_nodes()
 where the node list is generated.
2. Removes the "unavail_node_str" argument to select_nodes()
as it is no longer useful. This string originally was originally
generated once at the start of the job scheduling logic for all jobs,
but since each job can have a different set of
 unavailable nodes (dedicated to user, group, etc.)
so the same string for all jobs can be misleading.

Bug 4932.

Prevent from wrongly returning, ESLURM_NODE_NOT_AVAIL from _pick_best_nodes when some jobs are using "--exclusive=user"

Bug 4932.