1. 10 May, 2012 1 commit
  2. 09 May, 2012 1 commit
    • Don Lipari's avatar
      Reset priority of system held jobs when dependency is satisfied · 9e9298b1
      Don Lipari authored
      The symptom is that SLURM schedules lower priority jobs to run when higher priority, dependent jobs have their dependencies satisfied.  This happens because dependent jobs still have a priority of 1 when the job queue is sorted in the schedule() function.  The proposed fix forces jobs to have their priority updated when their dependencies are satisfied.
      9e9298b1
  3. 04 May, 2012 1 commit
  4. 03 May, 2012 1 commit
    • Matthieu Hautreux's avatar
      Fix segv in slurmctld for job step with relative option · 9bb178c3
      Matthieu Hautreux authored
      Here is the way to reproduce it :
      [root@cuzco27 georgioy]# salloc -n64 -N4 --exclusive
      salloc: Granted job allocation 8
      [root@cuzco27 georgioy]#srun -r 0 -n 30 -N 2 sleep 300&
      [root@cuzco27 georgioy]#srun -r 1 -n 40 -N 3 sleep 300&
      [root@cuzco27 georgioy]# srun: error: slurm_receive_msg: Zero Bytes were transmitted or received
      srun: error: Unable to create job step: Zero Bytes were transmitted or received
      9bb178c3
  5. 02 May, 2012 1 commit
  6. 27 Apr, 2012 2 commits
  7. 26 Apr, 2012 2 commits
  8. 24 Apr, 2012 1 commit
  9. 23 Apr, 2012 2 commits
  10. 20 Apr, 2012 1 commit
  11. 18 Apr, 2012 1 commit
  12. 17 Apr, 2012 3 commits
  13. 12 Apr, 2012 1 commit
  14. 10 Apr, 2012 4 commits
  15. 09 Apr, 2012 1 commit
  16. 03 Apr, 2012 2 commits
  17. 02 Apr, 2012 2 commits
    • Morris Jette's avatar
      Fix in select/cons_res+topology+job with node range count · cd84134c
      Morris Jette authored
      The problem was conflicting logic in the select/cons_res plugin. Some of the code was trying to get the job the maximum node count in the range while other logic was trying to minimize spreading out of the job across multiple switches. As you note, this problem only happens when a range of node counts is specified and the select/cons_res plugin and the topology/tree plugin and even then it is not easy to reproduce (you included all of the details below).
      
      Quoting Martin.Perry@Bull.com:
      
      > Certain combinations of topology configuration and srun -N option produce
      > spurious job rejection with "Requested node configuration is not
      > available" with select/cons_res. The following example illustrates the
      > problem.
      >
      > [sulu] (slurm) etc> cat slurm.conf
      > ...
      > TopologyPlugin=topology/tree
      > SelectType=select/cons_res
      > SelectTypeParameters=CR_Core
      > ...
      >
      > [sulu] (slurm) etc> cat topology.conf
      > SwitchName=s1 Nodes=xna[13-26]
      > SwitchName=s2 Nodes=xna[41-45]
      > SwitchName=s3 Switches=s[1-2]
      >
      > [sulu] (slurm) etc> sinfo
      > PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
      > ...
      > jkob         up   infinite      4   idle xna[14,19-20,41]
      > ...
      >
      > [sulu] (slurm) etc> srun -N 2-4 -n 4 -p jkob hostname
      > srun: Force Terminated job 79
      > srun: error: Unable to allocate resources: Requested node configuration is
      > not available
      >
      > The problem does not occur with select/linear, or topology/none, or if -N
      > is omitted, or for certain other values for -N (for example, -N 4-4 and -N
      > 2-3 work ok). The problem seems to be in function _eval_nodes_topo in
      > src/plugins/select/cons_res/job_test.c. The srun man page states that when
      > -N is used, "the job will be allocated as many nodes as possible within
      > the range specified and without delaying the initiation of the job."
      > Consistent with this description, the requested number of nodes in the
      > above example is 4 (req_nodes=4).  However, the code that selects the
      > best-fit topology switches appears to make the selection based on the
      > minimum required number of nodes (min_nodes=2). It therefore selects
      > switch s1.  s1 has only 3 nodes from partition jkob. Since this is fewer
      > than req_nodes the job is rejected with the "node configuration" error.
      >
      > I'm not sure where the code is going wrong.  It could be in the
      > calculation of the number of needed nodes in function _enough_nodes.  Or
      > it could be in the code that initializes/updates req_nodes or rem_nodes. I
      > don't feel confident that I understand the logic well enough to propose a
      > fix without introducing a regression.
      >
      > Regards,
      > Martin
      cd84134c
    • Morris Jette's avatar
      Use site maximum for option switch wait time. · 2581fe62
      Morris Jette authored
      When the optional max_time is not specified for --switches=count, the site
      max (SchedulerParameters=max_switch_wait=seconds) is used for the job.
      Based on patch from Rod Schultz.
      2581fe62
  18. 30 Mar, 2012 1 commit
  19. 29 Mar, 2012 2 commits
    • Mark Nelson's avatar
      Added CrpCPUMins to the output of sshare -l for those using hard limit · d1ae3d81
      Mark Nelson authored
      accounting.  Work contributed by Mark Nelson.
      d1ae3d81
    • Morris Jette's avatar
      Fix in select/cons_res+topology+job with node range count · f64b29a2
      Morris Jette authored
      The problem was conflicting logic in the select/cons_res plugin. Some of the code was trying to get the job the maximum node count in the range while other logic was trying to minimize spreading out of the job across multiple switches. As you note, this problem only happens when a range of node counts is specified and the select/cons_res plugin and the topology/tree plugin and even then it is not easy to reproduce (you included all of the details below).
      
      Quoting Martin.Perry@Bull.com:
      
      > Certain combinations of topology configuration and srun -N option produce
      > spurious job rejection with "Requested node configuration is not
      > available" with select/cons_res. The following example illustrates the
      > problem.
      >
      > [sulu] (slurm) etc> cat slurm.conf
      > ...
      > TopologyPlugin=topology/tree
      > SelectType=select/cons_res
      > SelectTypeParameters=CR_Core
      > ...
      >
      > [sulu] (slurm) etc> cat topology.conf
      > SwitchName=s1 Nodes=xna[13-26]
      > SwitchName=s2 Nodes=xna[41-45]
      > SwitchName=s3 Switches=s[1-2]
      >
      > [sulu] (slurm) etc> sinfo
      > PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
      > ...
      > jkob         up   infinite      4   idle xna[14,19-20,41]
      > ...
      >
      > [sulu] (slurm) etc> srun -N 2-4 -n 4 -p jkob hostname
      > srun: Force Terminated job 79
      > srun: error: Unable to allocate resources: Requested node configuration is
      > not available
      >
      > The problem does not occur with select/linear, or topology/none, or if -N
      > is omitted, or for certain other values for -N (for example, -N 4-4 and -N
      > 2-3 work ok). The problem seems to be in function _eval_nodes_topo in
      > src/plugins/select/cons_res/job_test.c. The srun man page states that when
      > -N is used, "the job will be allocated as many nodes as possible within
      > the range specified and without delaying the initiation of the job."
      > Consistent with this description, the requested number of nodes in the
      > above example is 4 (req_nodes=4).  However, the code that selects the
      > best-fit topology switches appears to make the selection based on the
      > minimum required number of nodes (min_nodes=2). It therefore selects
      > switch s1.  s1 has only 3 nodes from partition jkob. Since this is fewer
      > than req_nodes the job is rejected with the "node configuration" error.
      >
      > I'm not sure where the code is going wrong.  It could be in the
      > calculation of the number of needed nodes in function _enough_nodes.  Or
      > it could be in the code that initializes/updates req_nodes or rem_nodes. I
      > don't feel confident that I understand the logic well enough to propose a
      > fix without introducing a regression.
      >
      > Regards,
      > Martin
      f64b29a2
  20. 28 Mar, 2012 1 commit
  21. 27 Mar, 2012 3 commits
  22. 26 Mar, 2012 1 commit
  23. 23 Mar, 2012 1 commit
    • Morris Jette's avatar
      Fix bug in GRES with CPU binding · 4f875d9f
      Morris Jette authored
      Fix bug in allocating GRES that are associated with specific CPUs. In some
      cases the code allocated first available GRES to job instead of allocating
      GRES accessible to the specific CPUs allocated to the job.
      4f875d9f
  24. 22 Mar, 2012 1 commit
  25. 21 Mar, 2012 3 commits