[bug,legacy feature] FILE_JOBS_CONF doesn't warn about invalid files, and tries to copy valid files to a relative directory to PWD
( This is a template for bug reports. Please fill in the relevant information and delete the rest. )
Hello @dbeltran and @bdepaula,
Autosubmit Version
master
branch, and probably latest released PyPI version, 4.0.105.
Expid affected(If applicable)
Which task has issues? Where is the log(If applicable)
Summary
Stella has a testing workflow that she created for EDITO using FILE_JOBS_CONF
, following our documentation https://autosubmit.readthedocs.io/en/latest/userguide/configure/develop_a_project.html#expdef-configuration.
I followed the same steps and reproduced the errors she had:
- If you use something like
autosubmit/jobs.conf
, Autosubmit won't complain about a non-YAML file used - If you use a non-existent file, Autosubmit doesn't log any warnings. The code checks if the file exists, if not... it simply continues the configuration loading/command execution.
- If you use a valid file, it still fails when
shutil.copyfile
is called
(autosubmit4) kinow@ranma:/tmp$ autosubmit -lc DEBUG create a004 -np
/home/kinow/micromamba/envs/autosubmit4/bin/autosubmit:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('autosubmit==4.1.4')
Autosubmit is running with 4.1.4
Preparing .lock file to avoid multiple instances with same expid.
Using project folder: /home/kinow/autosubmit/a004/proj
Checking configuration files...
Expdef config file is correct
Platforms sections: OK
Jobs sections OK
Autosubmit general sections OK
Configuration files OK
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/autosubmit/bin/autosubmit", line 57, in main
return_value = Autosubmit.parse_args()
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 710, in parse_args
return Autosubmit.create(args.expid, args.noplot, args.hide, args.output, args.group_by, args.expand,
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 4678, in create
Autosubmit._create_project_associated_conf(
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 4604, in _create_project_associated_conf
shutil.copyfile(os.path.join(as_conf.get_project_dir(), as_conf.get_file_jobs_conf()),
File "/home/kinow/micromamba/envs/autosubmit4/lib/python3.9/shutil.py", line 266, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: 'autosubmit/jobs.yml'
[CRITICAL] An Unknown error occurred: [Errno 2] No such file or directory: 'autosubmit/jobs.yml'.
Please report it to Autosubmit Developers through Git
More info at https://autosubmit.readthedocs.io/en/master/troubleshooting/error-codes.html
Steps to reproduce
Create a project with autosubmit/jobs.yml
, with valid jobs.
Create a workflow and use a local project with
PROJECT:
PROJECT_DESTINATION: local_project
PROJECT_TYPE: local
PROJECT_FILES:
FILE_JOBS_CONF: autosubmit/jobs.yml
FILE_PROJECT_CONF: ''
JOB_SCRIPTS_TYPE: ''
RERUN:
RERUN: false
RERUN_JOBLIST: ''
ROOTDIR: /home/kinow/autosubmit/a004
Go to a directory like /tmp
, then run autosubmit -lc DEBUG create -np $expid
What is the current bug behavior?
It locates the jobs.yml
and tries to copy into $PWD/autosubmit/jobs.yml
(odd).
What is the expected correct behavior?
Not sure. It seems to be like this is legacy from AS3 and probably should be removed? I suggested Stella to use minimal
+ the CUSTOM CONFIG setting we use in DestinE.
Relevant logs and/or screenshots(if applicable)
Above.
Any other relevant information(if applicable)
()