Bug 1208

This patch provides a rewrite of how /proc/cpuinfo is parsed in common_jag.c, as the original code made the incorrect assumption that cpuinfo follows a sane format across architectures ;-)

The motivation for this patch is that the original code was producing stack smashing on a POWER7 running RHEL6.4 Red Hat adds -fstack-protector along with a lot of other protective CFLAGS when building RPMs. The code ran okay with -fno-stack-protector, but that is not the best work-around.

So, the relevant /proc/cpuinfo line on an Intel (Xeon X5675) system looks like:

cpu MHz                : 3066.915

Whereas the relevant line in a POWER7 system is

clock                : 3550.000000MHz

My patch replaces the assumption that the relevant number starts 11 characters into the string with another assumption: That the relevant number starts two characters after a colon in a string that matches (M|G)Hz.

All in all, the function has a few more calls, which may be a performance issue if it has to be called multiple times, but since the section I edited only gets evaluated if we don't know the cpu frequency, getting it right will actually result in fewer string operations and unnecessary opens of /proc/cpuinfo for systems likewise affected.

Finally, I also read the actual value into a double and multiply it up to the size indicated by the suffix, so we end up with KHz? It was unclear what the original code intended, since it matched both MHz and GHz, replaced the decimal point with a zero, and read the result as an int.

--
Chandler Wilkerson
Center for Research Computing
Rice University

This reverts commit 2c95e2d2.

Conflicts:
	src/plugins/select/alps/basil_interface.c

This is related to bug 1822.  It isn't clear why the code was taken out in
this commit in the first place and based off of commit 2e2de6a4 (which is
the reason for the conflict) we tried unsuccessfully to put it back.

It appears the only difference here is the addition of
always setting mppnppn = 1 instead of always to
job_ptr->details->ntasks_per_node when no ntasks is set.

This appears to only be an issue with salloc or sbatch as ntasks
is always set for srun.

Bug 1783

Prevent slurmctld abort on update of advanced reservation that contains no
    nodes.
bug 1814

srun command line of either --mem or --mem-per-cpu will override both the
SLURM_MEM_PER_CPU and SLURM_MEM_PER_NODE environment variables.
Without this change, salloc or sbatch setting --mem-per-cpu (or a
configuration of DefMemPerCPU) would over-ride the step's --mem
value.

change was made.

when removing a limit from an association on multiple clusters
at the same time.

to gain the correct limit when a parent account is root and you
remove a subaccount's limit which exists on the parent account.

Bug 1770

Bug 1798

Previous logic could fail to update some tasks of a job array for
  some fields.
bug 1777

bind to the interface of return of gethostname instead of any address on
the node which avoid RSIP issues in Cray systems.  This is most likely
useful in other systems as well.

Fix to job array update logic that can result in a task ID of 4294967294.
To reproduce:
$ sbatch --exclusive -a 1,3,5 tmp
Submitted batch job 11825
$ scontrol update jobid=11825_[3,4,5] timelimit=3
$ squeue
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
           11825_3     debug      tmp    jette PD       0:00      1 (None)
           11825_4     debug      tmp    jette PD       0:00      1 (None)
           11825_5     debug      tmp    jette PD       0:00      1 (None)
             11825     debug      tmp    jette PD       0:00      1 (Resources)
A new job array entry was created for task ID 4 and the "master" job
array record now has a task ID of 4294967294.
The logic with the bug was using the wrong variable in a test.
bug 1790

Bug 1799

Increase total backfill scheduler run time in stats_info_response_msg data
    structure from 32 to 64 bits in order to prevent overflow.

remove new capabilities added in comit ad9c2413
Leave the new logic only in version 15.08, which has related
performance improvements in the slurmctld agent code, see commit
53534f49

Modify slurmctld outgoing RPC logic to support more parallel tasks (up to
    85 RPCs and 256 pthreads; the old logic supported up to 21 RPCs and 256
    threads). This change can dramatically improve performance for RPCs
    operating on small node counts.
bug 1786

Correct "sdiag" backfill cycle time calculation if it yields locks. A
    microsecond value was being treated as a second value resulting in an
    overflow in the calcuation.
bug 1788

The slurmctld logic throttles some RPCs so that only one of them
can execute at a time in order to reduce contention for the job,
partition and node locks (only one of the effected RPCs can execute
at any time anyway and this lets other RPC types run). While an
RPC is stuck in the throttle function, do not count that thread
against the slurmctld thread limit.
but 1794

Changed spaces to tabs at start of lines.
Minor changes to some formatting.
Added the new files to the RPM (slurm.spec file).
Prevent memory leak of "l_name" variable if which_power_layout()
    function is called more than once.
Initialize "cpufreq" variable in powercap_get_cpufreq() function.
Array "tmp_max_watts_dvfs" could be NULL and used if "max_watts_dvfs" variable is
    NULL in powercap_get_node_bitmap_maxwatts_dvfs()
Variable "tmp_pcap_cpu_freq" could be used with uninitialized value in function
    _get_req_features()
Variable "tmp_max_watts" could be used with uninitialized value in function
    _get_req_features()
Array "tmp_max_watts_dvfs" could be used with uninitialized value in function
    _get_req_features()
Array "allowed_freqs" could be NULL and used if "node_record_count" variable is
    zero in powercap_get_job_nodes_numfreq()
Overwriting a memory buffer header (especially with different data types) is
    just asking for something bad to happen. This code from function
    powercap_get_job_nodes_numfreq():
			allowed_freqs = xmalloc(sizeof(int)*((int)num_freq+2));
			allowed_freqs[-1] = (int) num_freq;
Clean up memory on slurmctld shutdown

Add job record qos field and partition record allow_qos field.

This patch moves the QOS update of an existing job to be before the
partition update.  This ensures a new QOS value is the value used when
doing validations against things like a partition's AllowQOS and DenyQOS.

Currently if a two partitions have AllowQOS that do not share any QOS,
the order of updates prevents a job from being moved from one partition
to another using something like the following:

scontrol update job=<jobID> partition=<new part> qos=<new qos>

Correct task layout with CR_Pack_Node option and more than 1 CPU per task.
Previous logic would place one task per CPU launch too few tasks.
bug 1781

Backfill scheduler now considers OverTimeLimit and KillWait configuration
parameters to estimate when running jobs will exit. Initially the job's
end time is estimated based upon it's time limit. After the time limit
is reached, the end time estimate is based upon the OverTimeLimit and
KillWait configuration parameters.
bug 1774

Backfill scheduler: The configured backfill_interval value (default 30
    seconds) is now interpretted as a maximum run time for the backfill
    scheduler. Once reached, the scheduler will build a new job queue and
    start over, even if not all jobs have been tested.
bub 1774

Original patch from LLNL assumed Munge was installed, which would
  result in a build error if the Munge development package was
  not installed

Add association usage information to "scontrol show cache" command output.

When submitting a job with srun -n# the job may be allocated more than # because
the job was given the whole core or socket (eg. CR_CORE, CR_SOCKET). sacct
showed only what the step used and not the allocation. This commit shows the job
and the step if job and step cpus are different.