In etc/init.d/slurm move check for scontrol after sourcing
/etc/sysconfig/slurm. Patch from Andy Wettstein, University of Chicago.

This change makes the code consistent with the documentation.
Note that "bf_res=" will continue to be recognized for now.
Patch from Rod Schultz, Bull.

I have implemented the changes as you suggested:   using a "-dd" option to indicate that the display of the script is wanted, and setting both the "SHOW_DETAIL" and a new "SHOW_DETAIL2" flag.

Since "scontrol" can be run interactively as well,  I added a new "script" option to indicate that display of both the script and the details is wanted if the job is a batch job.

Here are the man page updates for "man scontrol".   For the "-d, --details" option:

       -d, --details
              Causes  the  show command to provide additional details where available.  Repeating the option more than
              once (e.g., "-dd") will cause the show job command to also list the batch script, if the job was a batch
              job.

For the interactive "details" option:

       details
              Causes  the  show  command  to provide additional details where available.  Job information will include
              CPUs and NUMA memory allocated on each node.  Note that on computers  with  hyperthreading  enabled  and
              SLURM  configured  to allocate cores, each listed CPU represents one physical core.  Each hyperthread on
              that core can be allocated a separate task, so a job's CPU count and task count  may  differ.   See  the
              --cpu_bind  and  --mem_bind  option  descriptions  in  srun man pages for more information.  The details
              option is currently only supported for the show job command. To also list the  batch  script  for  batch
              jobs, in addition to the details, use the script option described below instead of this option.

And for the new interactive "script" option:

       script Causes the show job command to list the batch script for batch jobs in addition to the  detail  informa-
              tion described under the details option above.

Attached are the patch file for the changes and a text file with the results of the tests I did to check out the changes.   The patches are against SLURM 2.4.0-rc1.

        -Don Albert-

compiling with --enable-debug

The purpose of this is so moab scripts and commands (such as 'checkjob')
have consistent access to the SUBMITHOST variable.

isn't up at the time the slurmctld starts, not running the
priority/multifactor plugin, and then the database is started up later.

Cray - Improve support for zero compute note resource allocations.
Partition used can now be configured with no nodes nodes.

The symptom is that SLURM schedules lower priority jobs to run when higher priority, dependent jobs have their dependencies satisfied.  This happens because dependent jobs still have a priority of 1 when the job queue is sorted in the schedule() function.  The proposed fix forces jobs to have their priority updated when their dependencies are satisfied.

The symptom is that SLURM schedules lower priority jobs to run when higher priority, dependent jobs have their dependencies satisfied.  This happens because dependent jobs still have a priority of 1 when the job queue is sorted in the schedule() function.  The proposed fix forces jobs to have their priority updated when their dependencies are satisfied.

developments.

Here is the way to reproduce it :
[root@cuzco27 georgioy]# salloc -n64 -N4 --exclusive
salloc: Granted job allocation 8
[root@cuzco27 georgioy]#srun -r 0 -n 30 -N 2 sleep 300&
[root@cuzco27 georgioy]#srun -r 1 -n 40 -N 3 sleep 300&
[root@cuzco27 georgioy]# srun: error: slurm_receive_msg: Zero Bytes were transmitted or received
srun: error: Unable to create job step: Zero Bytes were transmitted or received

cpus in task/cgroup plugin

Cray - Add support for zero compute note resource allocation to run batch
script on front-end node with no ALPS reservation. Useful for pre- or post-
processing. NOTE: The partition must be configured with MinNodes=0.

batch jobs.

Sinfo output format of "%P" now prints "*" after default partition even if
no field width is specified (previously included "*" only if no field width
was specified. Added output format of "%R" to print partition name only
without identifying the default partition with "*").

Previously the code would come up with how much memory a PE should have
instead of the memory a node should have.

Mark Nelson.

larger than midplane jobs.

Add support for new SchedulerParameters of bf_max_job_user, maximum number
of jobs to attempt backfilling per user. Work by Bjørn-Helge Mevik,
University of Oslo.

Fix sched/wiki2 to support job account name, gres, partition name, wckey,
or working directory that contains "#" (a job record separator). Without
this patch, the parsing will probably stop once reaching the "#".

and time limit where it was previously set by an admin.

slurmdbd accounting and running large amounts of jobs (>50 sec).  Job
information could be corrupted before it had a chance to reach the DBD.