Task prologs could set or modify this, so wait to create the
directory until after they've finished.

Bug 5367.

After changes to slurm_send_only_node_msg(), this message is much
more likely to appear on systems with overloaded interconnects since
that connection handling code may end up retransmitting messages
that were actually received (but that the transmit side could not
verify were delivered successfully).

As the error() message stated, this isn't actually an error, and
the code will proceed happily past this point. So drop the debug
level, and remove the surrealist "this is not an error" part.

Bug 5164.

There are subtle issues involved in treating a TCP transmission
as a unidirectional message delivery layer.

The original code path looks like: connect(), write(), close().
But Linux handles the write() and close() asynchronously behind the
scenes, and does not block until that write() has been ACK'd by the
remote end. So the write() and close() may succeed, even with data
still in flight. A communication error - and message loss - would
have been silently ignored, leading to unreliable message transmission.

Worse yet, one side of the connection would believe it sent the message,
while the receive side swears it never saw the packets. This leads to
infrequent and yet seemingly impossible data loss, and a very tough
bug to chase down.

This teardown code tries to force the connection to shut down in an
orderly manner, giving Slurm a chance to catch a connection problem
and the upstream calling path an opportunity to retransmit.

This teardown code is based on an approach described in Section 7.5
of "UNIX Network Programming" Volume 1 (Third Edition), specifically
the subsection regarding SO_LINGER. (And also covers why SO_LINGER is
not sufficent to prevent this issue.)

Bug 5164.

Bug 5164.

The getopt format string needs to handle an option here,
and the --help output had not been corrected after 99b2c4e8.

Bug 5522.

Bug 5070

Bug 5468

This is a backport of commit cefc9ec1.

scheduling cycle.  Primarily caused by EnforcePartLimits=ALL.

Bug 5452

Update slurm.spec and slurm.spec-legacy as well.

The lower limit of 1024 may be too short for srun with large-scale
jobs, and lead to problems processing task completion messages in a
timely fashion.

Rather than adjust that, unify the two separate macros into
SLURM_DEFAULT_LISTEN_BACKLOG with the higer 4096 value.

Bug 5164.

Without Delegate=yes, systemd will "fix" the cgroup hierarchies whenever
'systemctl daemon-reload' is called, which will then remove any
restrictions placed on memory or device access for a given job.

This is a problem especially since 'systemctl daemon-reload' may be called
automatically by rpm/yum or a variety of config file mangers, leading to
jobs escaping from slurmd/slurmstepd's control.

This setting should work for systemd versions >= 205.
https://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/

Bug 5292.

When a job with time_end=0 and TRES null exists from an association that is
currently inside a reservation, the hourly rollup segfaults.

Bug 5143

As reported by Avalon Johnson on slurm-users
https://groups.google.com/forum/#!topic/slurm-users/BsMQ7Uk1PLw
Bug 5287.

srun was already fixed in b7053bda (Bug 3294).

Bug 5126

'have_innodb' is deprecated.

Bug 5317.

When printing arrays in squeue and setting the SLURM_BITSTR_LEN variable to 0
or to NULL, the length of the output defaulted to 64, when the documentation
says it will default to "unlimited". This patch fixes this situation.

Bug 5440

Logic was switched around in 17.11, enable_user_top is now the
correct option.

Bug 5165.

This is not working reliably even when setting
SchedulerParameters=enable_hetero_steps and/or using OpenMPI with Slurm's
mpi/pmi2, as it was previously documented.

Bug 5309.

Documented, and code reads as needing, the node lock. But these
were incorrectly set as the job locks.

Bug 5394.

Needs the job write lock, as it may change job status not just node
status. Especially after commit 33e352a6.

Bug 5406.

Previously, usage was just CPU-seconds. However, since
TresBillingWeights has been added, usage gets calculated from that, or
it's just CPU-seconds if TresBillingWeights isn't defined.

Bug 5434.

Add errno to info message in the SlurmDBD log, and pass the actual
errno back to the sacctmgr process so the user can see it.

Bug 5152.

- avoid `abort()` when collective is failed
- added logging of coll details for fail cases

Bug 5067

Note, this is setting it up so we can use defunct functions.  It will
probably need to be properly fixed in a future version so we don't
do this.

Bug 5098.

with preemption or when job requests a specific list of hosts.

Bug 5293.

Bug 5395

Bug 5390

Continuation of 923c9b37.

There is a delay in the cgroup system when moving a PID from one cgroup
to another. It is usually short, but if we don't wait for the PID to
move before removing cgroup directories the PID previously belonged to,
we could leak cgroups. This was previously fixed in the cpuset and
devices subsystems. This uses the same logic to fix the freezer
subsystem.

Bug 5082.

cpuset and devices subsystems have duplicate code to cleanup the cgroup
and prevent leaking cgroups by moving the process to the root cgroup and
waiting for it to be moved.

Move this duplicate code to a common function so it can be used later by
the freezer subsystem.

Bug 5082.

Bug 5227