1. 03 Jun, 2013 1 commit
    • Hongjia Cao's avatar
      restore max_nodes of desc to NO_VAL when checkpointing job · f82e0fb8
      Hongjia Cao authored
      We're having some trouble getting our slurm jobs to successfully
      restart after a checkpoint.  For this test, I'm using sbatch and a
      simple, single-threaded executable.  Slurm is 2.5.4, blcr is 0.8.5.
      I'm submitting the job using sbatch:
      
      $ sbatch -n 1 -t 12:00:00 bin/bowtie-ex.sh
      
      I am able to create the checkpoint and vacate the node:
      
      $ scontrol checkpoint create 137
      .... time passes ....
      $ scontrol vacate 137
      
      At that point, I see the checkpoint file from blcr in the current
      directory and the checkpoint file from Slurm
      in /var/spool/slurm-llnl/checkpoint.  However, when I attempt to
      restart the job:
      
      $ scontrol checkpoint restart 137
      scontrol_checkpoint error: Node count specification invalid
      
      In slurmctld's log (at level 7) I see:
      
      [2013-05-29T12:41:08-07:00] debug2: Processing RPC: REQUEST_CHECKPOINT(restart) from uid=*****
      [2013-05-29T12:41:08-07:00] debug3: Version string in job_ckpt header is JOB_CKPT_002
      [2013-05-29T12:41:08-07:00] _job_create: max_nodes == 0
      [2013-05-29T12:41:08-07:00] _slurm_rpc_checkpoint restart 137: Node count specification invalid
      f82e0fb8
  2. 30 May, 2013 1 commit
  3. 29 May, 2013 1 commit
  4. 23 May, 2013 4 commits
  5. 22 May, 2013 2 commits
  6. 18 May, 2013 1 commit
  7. 16 May, 2013 2 commits
  8. 14 May, 2013 1 commit
  9. 13 May, 2013 1 commit
  10. 08 May, 2013 2 commits
  11. 02 May, 2013 2 commits
  12. 01 May, 2013 4 commits
  13. 30 Apr, 2013 1 commit
  14. 29 Apr, 2013 3 commits
  15. 26 Apr, 2013 3 commits
  16. 25 Apr, 2013 1 commit
  17. 23 Apr, 2013 2 commits
  18. 19 Apr, 2013 1 commit
  19. 18 Apr, 2013 1 commit
  20. 17 Apr, 2013 3 commits
  21. 16 Apr, 2013 1 commit
  22. 12 Apr, 2013 1 commit
  23. 11 Apr, 2013 1 commit