Moe,

After doing more extensive testing, I came to realize
that we had made a bad basic assumption. We believed
that the user's environment should only be what was
sent in the file via the --export-file option.

However, that broke the previous behavior, especially
in regard to Moab jobs. It also caused the SLURM
defined environment variables to be lost as well.

This patch will enable the correct behavior for Moab
on top of SLURM whne using the --export-file option,
but the behavior is less that pefect for using it
stand alone with sbatch. When using the option with
sbatch as a user, the file environment is read in,
and then when the env_array_merge is made, some
variables may get overwritten. This is good for
the SLURM and MPI vairables, but not so good for
others., The problem is trying to reconcile two
sources of environment is very problematic.

I also added a caveat in the man page.

I made changes in my branch of SchedMD SLURM
for 2.3, here is the patch.

Phil

Fix race condition that could generate job_cnt_comp underflow errors on
front-end architectures (Cray or IBM BlueGene systems).

just the accounting plugins.

It was found if the conditions were right that this could cause the state
to be set to 0 if it was being set to RUNNING at the same time in
node_scheduler.c in the slurmctld.  It probably would never happen on a
real system but this makes the problem go away.

Fix for possible invalid memory reference in slurmctld in job dependency
logic. Patch from Carles Fenoy (Barcelona Supercomputer Center).

jobs finish

Insure that BlueGene system has partition configured with Shared=FORCE
whenever Dynmaic blocks are configured before trying to create sub-
midplane blocks.

all jobs would be returned even if the flag was set.
Patch from Bill Brophy, Bull.

We replaced references to "pipe" with a more generic "file descriptor".
We also replaced a while loop in env.c with a for loop.

Fix bug in --switch option with topology resulting in bad switch count use.
Patch from Alejandro Lucero Palau (Barcelona Supercomputer Center).

Avoid calling jobacct_gather_g_getinfo() until there is data to read from the socket.

In current version of slurm initscript, a stop action returns a
non null exit code as slurmstatus exit code is directly used and
the daemons are stopped.
Ensure that when called from slurmstop, slurmstatus error code is
reversed to correctly match the attended error code of the stop
stage.

When multiple slurmd is enabled, there is no way in the prolog
and epilog scripts associated to the slurmd daemons to determine
which node, either virtual or physical is associated to the
execution. SLURMD_NODENAME will provide that information, that is
to say the name of the virtual or physical SLURM node at the origin
of the prolog/epilog execution.

When cpuset is used as the task/affinity backend, a common root
directory is used to group every cpuset subdirectories created
by the plugin. With multiple slurmd support, this result in all
the virtual nodes executing on a same physical node sharing the
same cpuset directories. This patch make sure that every virtual
node has its own root directory in the cpuset hierarchy.