Improve logs when AS is creating the wrappers
Hello @dbeltran and @bdepaula,
Autosubmit Version
4.1.3
Expid affected(If applicable)
a1bx
Which task has issues? Where is the log(If applicable)
Summary
When a big experiment (3000 chunks) uses wrappers, there is a "Hang" when there are READY jobs the logs are stopped in this point:
3651 of 3653 jobs remaining (16:53)
No jobs to check for platform LUMI
No jobs to check for platform local
Checking 1 jobs for platform LUMI-LOGIN
Job a1bx_REMOTE_SETUP is COMPLETED
The ASLOGS run are like this:
2024-05-22 16:53:58,090 Setting job: a1bx_20200120_fc0_3649_TRANSFER status to: READY (all parents completed)...
2024-05-22 16:53:58,090 Setting job: a1bx_20200120_fc0_3650_TRANSFER status to: READY (all parents completed)...
2024-05-22 16:53:58,092 Update finished
2024-05-22 16:53:58,098 Saving JobList: /appl/AS/AUTOSUBMIT_DATA/a1bx/pkl/job_list_a1bx.pkl.tmp
2024-05-22 16:53:58,192 JobList saved in /appl/AS/AUTOSUBMIT_DATA/a1bx/pkl/job_list_a1bx.pkl
2024-05-22 16:53:58,216 Number of jobs available: 16
2024-05-22 16:53:58,223 Number of jobs ready: 3650
2024-05-22 16:53:58,237 Jobs ready for LUMI: 3650
So I thought that there was a problem but AS was doing some work in the background and 4min later it submitted my wrapped jobs (thx @dbeltran !), I think it would be useful to print something. Thanks!!
()