Bug 5682 and 4584

Backport of ba07a6e09b6071

other than knl.

Bug 5603

Bug 5603

This patch also simplifies the code using bit_cnt instead of
_bitstr_bits().  This isn't really necessary, but does look cleaner.

Bug 6216

It was using the local cluster, sending the query to the wrong table

Bug 6159

Bug 6237

if OverTimeLimit is Unlimited.

Bug 6093

Only print a warning for 18.08. If a user has SLURM_MEM_PER_CPU or
SLURM_MEM_PER_NODE environment variables set for some reason this
situation could be happening by accident, and we don't want to prevent
the srun command from launching steps at this point.

Bug 6058.

Bug 5648.

The Linux kernel default hard limit of 4096 for the number of file
descriptors is quite small. Debian/Ubuntu have for a long time
overridden this, increasing it to 1M. Recently systemd also bumped the
default to 512k.

https://github.com/systemd/systemd/blob/master/NEWS

https://github.com/systemd/systemd/pull/10244

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/ZN5TK3D6L7SE46KGXICUKLKPX2LQISVX/

https://github.com/systemd/systemd/commit/09dad04c49cae3ad2b319c9b4e7773fedd34309a

Here the limits are increased as follows:

- slurmd: 128k; some workloads like Hadoop/Spark need a lot of fd's,
  and recommend that the limit is increased to at least 64k.

- slurmctld: 64k; per the Slurm high throughput and big system guides
  which recommend a file-max of at least 32k.

- slurmdbd: 64k, matching slurmctld, though slurmdbd shouldn't need
  that many fd's, bumping the limit shouldn't hurt either.

Bug 6171

Bug 5248

Bug 6055

Add the following slurmctld return codes to the lua plugin:

  ESLURM_ACCESS_DENIED
  ESLURM_ACCOUNTING_POLICY
  ESLURM_INVALID_NODE_COUNT
  ESLURM_JOB_MISSING_SIZE_SPECIFICATION
  ESLURM_MISSING_TIME_LIMIT

Bug 6055

Backups already run it when dropping to backup.

Bug 6098.

Also throw an error message within stepd_available() if the nodename
is not set or cannot be inferred correctly.

Bug 5399.

Bug 6120

Bug 6155

Bug 6155

When bf_continue is set, and locks are released during a backfill cycle,
other operations can make new resorces available while part way through
the queue. When backfill continues the cycle and evaluates new jobs, it
may allocate some of these newly available resources to lower priority jobs,
rather than to higher priority jobs that were already considered in this
backfill cycle.

This patch introduces bf_ignore_newly_avail_nodes to SchedulerParameters
to solve this issue. This option will ignore nodes made available when
the backfill scheduler yields when resuming the backfill cycle.

Bug 5279.

slurmd yet delivering it's TRES list.

Bug 6122

Co-authored-by: Marshall Garey <marshall@schedmd.com>

Bug 6008

then an error is generated if more than one of those specifications
contains KNL NUMA or MCDRAM modes.

Bug 5846

are down nodes.

Bug 5846

NODE_SET_REBOOT to continue.

Bug 5846

node change when possible.

Bug 5846

Plugins reading in their own config files rely on the SLURM_CONF
environment variable pointing to the appropriate directory,
otherwise they will fall back to the build in sysconfdir path.

Set the environment variable early enough so that the -f flag
operates correctly, but not before conf->conffile has definitely
been set. Remove the setenv call that happens before the first
slurmstepd is fork()'d as it is now redundant.

Bug 4774.

sbatch sets these, but salloc did not. This should make srun behavior
between the two consistent.

Bug 3861.

time that wasn't existent instead of just updating lines that have time
with a lesser time.

Slurm is going to replace internally.

Bug 5800

Bug 6121

Bug 6008

Bug 6077

node system.

Bug 6037

Bug 6037

Bug 5983

In case of the error code paths (like collective timeout) it is possible
that a callback provided by PMIx will be called twice leading to a
segmentation fault.
This commit fixes it by properly accounting callback invocations.

Bug 5983

Bug 5935

This could have caused core dumps if communication failed for one
reason or another.

Signed-off-by: Boris Karasev <karasev.b@gmail.com>

Bug 5935

This patch does 2 things:
1. When a step fails on some node, then mark it as complete on those
   nodes. This is needed so that when the step ends on the other
   nodes, slurmctld recognized the step as completely done.
2. If the step does not have the --no-kill option set, then when some
   node fails, send a request to terminate the step on ALL of its nodes.

Bug 5805