files at tmp not created

Autosubmit Version

v4.1.9

Expid affected(If applicable)

a7wv

Summary

I am trying to run autosubmit at the hub sending jobs to bscearth000. The problem is that the files in the tmp directory are not being created when I use the bscearth00 platform, and they are needed to run my job.

I have done it with a simple job without any dependency on the tmp files and it has worked.

I have also run the same job locally and it has worked.

When I run autosubmit inspect these files are created, but if I run autosubmit create they disappear.

Steps to reproduce

Create an experiment with:

 $ autosubmit expid -d "my first auto-phenomena test" -H "Marenostrum5" -min --git_as_conf conf/bootstrap --git_repo https://earth.bsc.es/gitlab/nextgeneu/internal/auto-phenomena --git_branch 10-platforms
 $ autosubmit create EXPID
 $ autosubmit run EXPID

Remember to change the user and scratch_dir on the bscearth platform at /esarchive/autosubmit/EXPID/proj/git_project/conf/platform.yml

What is the current bug behavior?

The file /esarchive/autosubmit/a7wv/tmp/ENTSOE_unit_2024102800_fc0_1_DOWNLOAD_ENTSOE_UNIT is not created.

What is the expected correct behavior?

The file /esarchive/autosubmit/a7wv/tmp/ENTSOE_unit_2024102800_fc0_1_DOWNLOAD_ENTSOE_UNIT created.

Relevant logs and/or screenshots(if applicable)

This is the error:

Traceback (most recent call last):
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/platforms/paramiko_platform.py", line 386, in send_file
    self._ftpChannel.put(local_path, remote_path)
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/Python/3.9.6-GCCcore-11.2.0/lib/python3.9/site-packages/paramiko/sftp_client.py", line 758, in put
    with open(localpath, "rb") as fl:
FileNotFoundError: [Errno 2] No such file or directory: '/esarchive/autosubmit/a7wv/tmp/ENTSOE_unit_2024102800_fc0_1_DOWNLOAD_ENTSOE_UNIT'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/job/job_packages.py", line 203, in submit
    self._send_files()
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/job/job_packages.py", line 246, in _send_files
    self.platform.send_file(os.path.join(self._tmp_path, full_path))
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/platforms/paramiko_platform.py", line 390, in send_file
    raise AutosubmitError('Can not send file {0} to {1}'.format(os.path.join(
log.log.AutosubmitError:  

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/platforms/platform.py", line 308, in submit_ready_jobs
    package.submit(as_conf, job_list.parameters, inspect, hold=hold)
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/job/job_packages.py", line 209, in submit
    raise AutosubmitCritical("Error while submitting jobs: {0}".format(e), 7013)
log.log.AutosubmitCritical:  

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/autosubmit.py", line 2229, in run_experiment
    Autosubmit.submit_ready_jobs(as_conf, job_list, platforms_to_test, packages_persistence, hold=False)
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/autosubmit.py", line 2516, in submit_ready_jobs
    save_1, failed_packages, error_message, valid_packages_to_submit, any_job_submitted = platform.submit_ready_jobs(as_conf,
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/platforms/platform.py", line 346, in submit_ready_jobs
    raise AutosubmitCritical(e.message, e.code, e.trace)
log.log.AutosubmitCritical:  

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/EGG-INFO/scripts/autosubmit", line 59, in main
    return_value = Autosubmit.parse_args()
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/autosubmit.py", line 707, in parse_args
    return Autosubmit.run_experiment(args.expid, args.notransitive,args.start_time,args.start_after, args.run_only_members, args.profile)
  File "/shared/earth/easybuild/rocky/8.4/x86_64/software/autosubmit/4.1.9-foss-2021b-Python-3.9.6/lib/python3.9/site-packages/autosubmit-4.1.9-py3.9.egg/autosubmit/autosubmit.py", line 2340, in run_experiment
    raise AutosubmitCritical(e.message, e.code, e.trace)
log.log.AutosubmitCritical:  

 [CRITICAL] Error while submitting jobs:   [eCode=7013]
More info at https://autosubmit.readthedocs.io/en/master/troubleshooting/error-codes.html

Thank you very much :)

cc: @agayayav ()

Edited Oct 28, 2024 by lherrero