Error in autosubmit create with new jobs.yml file
Hello @dbeltran and @bdepaula,
Autosubmit Version
4.1-beta
Expid affected(If applicable)
a0g7
Which task has issues? Where is the log(If applicable)
log:
Click to expand
[froura@climatedt-wf conf]$ autosubmit create a0g7 -v
Autosubmit is running with 4.1.0
Preparing .lock file to avoid multiple instances with same expid.
Using project folder: /appl/AS/AUTOSUBMIT_DATA/a0g7/proj
Checking configuration files...
Expdef config file is correct
Platforms sections: OK
Jobs sections OK
Autosubmit general sections OK
Configuration files OK
Loading parameters...
Creating the jobs list...
Loading JobList
Loading JobList
Creating jobs...
Load finished
Adding dependencies to the graph..
Traceback (most recent call last):
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/autosubmit.py", line 4693, in create
job_list.generate(as_conf,date_list, member_list, num_chunks, chunk_ini, parameters, date_format,
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/job/job_list.py", line 260, in generate
self._add_dependencies(date_list, member_list, chunk_list, self._dic_jobs)
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/job/job_list.py", line 330, in _add_dependencies
self._apply_jobs_edge_info(job, dependencies)
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/job/job_list.py", line 844, in _apply_jobs_edge_info
if self.graph.nodes[parent]['job'].section in filters_to_apply_by_section.keys():
KeyError: 'job'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/EGG-INFO/scripts/autosubmit", line 37, in main
return_value = Autosubmit.parse_args()
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/autosubmit.py", line 701, in parse_args
return Autosubmit.create(args.expid, args.noplot, args.hide, args.output, args.group_by, args.expand,
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/autosubmit.py", line 4799, in create
raise AutosubmitCritical(str(e), 7070)
log.log.AutosubmitCritical:
[CRITICAL] 'job' [eCode=7070]
More info at https://autosubmit.readthedocs.io/en/master/troubleshooting/error-codes.htmlSummary
Experiment failed to be created [CRITICAL] 'job' [eCode=7070]
Steps to reproduce
Create experiment with this new jobs.yml
Click to expand
JOBS:
APP:
DEPENDENCIES: OPA
FILE: templates/application.sh,templates/only_lra.yaml
FOR:
DEPENDENCIES:
- OPA_AQUA_1:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
- OPA_MHM_1:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
OPA_MHM_2:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
- OPA_WILDFIRES_FWI_1:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
OPA_WILDFIRES_FWI_2:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
OPA_WILDFIRES_FWI_3:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
OPA_WILDFIRES_FWI_4:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
OPA_WILDFIRES_FWI_5:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
- OPA_URBAN_1:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
NAME: '%RUN.APP_NAMES%'
SPLITS: '[31, 31, 31, 31]'
PLATFORM: '%DEFAULT.HPCARCH%'
RUNNING: chunk
WALLCLOCK: 00:05
DN:
DEPENDENCIES:
DN:
SPLITS_FROM:
all:
SPLITS_TO: previous
SIM:
STATUS: RUNNING
FILE: templates/dn.sh,conf/mother_request.yml
PLATFORM: '%DEFAULT.HPCARCH%-login'
RUNNING: chunk
SPLITS: 31
WALLCLOCK: 00:15
INI:
DEPENDENCIES: REMOTE_SETUP
FILE: templates/ini.sh
PLATFORM: '%DEFAULT.HPCARCH%-login'
RUNNING: member
WALLCLOCK: 00:30
LOCAL_SETUP:
FILE: templates/local_setup.sh
PLATFORM: LOCAL
RUNNING: once
OPA:
DEPENDENCIES:
DN:
SPLITS_FROM:
all:
SPLITS_TO: '[1:%JOBS.DN.SPLITS%]*\\1'
FILE: templates/opa.sh
FOR:
NAME: '%RUN.OPA_NAMES%'
SPLITS: '[31, 31, 31, 31, 31, 31, 31, 31, 31]'
PLATFORM: '%DEFAULT.HPCARCH%'
RUNNING: chunk
REMOTE_SETUP:
DEPENDENCIES: SYNCHRONIZE
FILE: templates/remote_setup.sh,templates/fdb/config.yaml
PLATFORM: '%DEFAULT.HPCARCH%-login'
RUNNING: once
WALLCLOCK: 02:00
SIM:
DEPENDENCIES: INI SIM-1
FILE: <to-be-replaced-by-user-conf>
PLATFORM: '%DEFAULT.HPCARCH%'
WALLCLOCK: 00:30
running: chunk
SYNCHRONIZE:
DEPENDENCIES: LOCAL_SETUP
FILE: templates/synchronize.sh
PLATFORM: LOCAL
RUNNING: once
RUN:
APP_NAMES:
- AQUA
- MHM
- WILDFIRES_FWI
- URBAN
OPA_NAMES:
- aqua_1
- mhm_1
- mhm_2
- wildfires_fwi_1
- wildfires_fwi_2
- wildfires_fwi_3
- wildfires_fwi_4
- wildfires_fwi_5
- urban_1
What is the current bug behavior?
experiment not created
What is the expected correct behavior?
experiment created successfully
Relevant logs and/or screenshots(if applicable)
Click to expand
[froura@climatedt-wf conf]$ autosubmit create a0g7 -v
Autosubmit is running with 4.1.0
Preparing .lock file to avoid multiple instances with same expid.
Using project folder: /appl/AS/AUTOSUBMIT_DATA/a0g7/proj
Checking configuration files...
Expdef config file is correct
Platforms sections: OK
Jobs sections OK
Autosubmit general sections OK
Configuration files OK
Loading parameters...
Creating the jobs list...
Loading JobList
Loading JobList
Creating jobs...
Load finished
Adding dependencies to the graph..
Traceback (most recent call last):
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/autosubmit.py", line 4693, in create
job_list.generate(as_conf,date_list, member_list, num_chunks, chunk_ini, parameters, date_format,
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/job/job_list.py", line 260, in generate
self._add_dependencies(date_list, member_list, chunk_list, self._dic_jobs)
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/job/job_list.py", line 330, in _add_dependencies
self._apply_jobs_edge_info(job, dependencies)
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/job/job_list.py", line 844, in _apply_jobs_edge_info
if self.graph.nodes[parent]['job'].section in filters_to_apply_by_section.keys():
KeyError: 'job'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/EGG-INFO/scripts/autosubmit", line 37, in main
return_value = Autosubmit.parse_args()
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/autosubmit.py", line 701, in parse_args
return Autosubmit.create(args.expid, args.noplot, args.hide, args.output, args.group_by, args.expand,
File "/appl/AS/4.1.0-beta/lib/python3.9/site-packages/autosubmit-4.1.0-py3.9.egg/autosubmit/autosubmit.py", line 4799, in create
raise AutosubmitCritical(str(e), 7070)
log.log.AutosubmitCritical:
[CRITICAL] 'job' [eCode=7070]
More info at https://autosubmit.readthedocs.io/en/master/troubleshooting/error-codes.htmlAny other relevant information(if applicable)
In the Error key, 7070 suggests opening an issue in this repo. The error may come from a missformat of the jobs.yml file, attached above.
Related with workflow issue: https://earth.bsc.es/gitlab/digital-twins/de_340/workflow/-/issues/153
()