Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • autosubmit autosubmit
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 338
    • Issues 338
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 21
    • Merge requests 21
  • Deployments
    • Deployments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth SciencesEarth Sciences
  • autosubmitautosubmit
  • Issues
  • #1255
Closed
Open
Issue created Mar 12, 2024 by Bruno de Paula Kinoshita@bdepaulaMaintainer

[bug] Bug with CHECK: on_submission with horizontal wrappers

@dbeltran , just creating an issue as this was reported by @agayayav in the DestinE workflow repository.

Here's what I did to reproduce the bug with the master branch, and affects latest version 4.1.2:

(autosubmit4) bdepaula@bsces107921:~/Development/python/workspace/autosubmit$ git log -n 1
commit 75464cc8728cf8f9b768fb47ff4d9e5f03663251 (HEAD -> master, upstream/master, upstream/HEAD)
Author: Daniel Beltran Mora <daniel.beltran@bsc.es>
Date:   Mon Mar 11 15:01:27 2024 +0100

    changelog
# File: minimal.yml
CONFIG:
  AUTOSUBMIT_VERSION: "4.1.2"
  MAXWAITINGJOBS: 20
  TOTALJOBS: 20
  SAFETYSLEEPTIME: 10
  RETRIALS: 0
MAIL:
  NOTIFICATIONS: False
  TO:
STORAGE:
  TYPE: pkl
  COPY_REMOTE_LOGS: true
DEFAULT:
  EXPID: "a005"
  HPCARCH: "local"
EXPERIMENT:
  DATELIST: '20000101'
  MEMBERS: fc0
  CHUNKSIZEUNIT: month
  CHUNKSIZE: '4'
  NUMCHUNKS: '2'
  CHUNKINI: ''
  CALENDAR: standard
PROJECT:
  PROJECT_TYPE: none
  PROJECT_DESTINATION: ''
GIT:
  PROJECT_ORIGIN: ''
  PROJECT_BRANCH: ''
  PROJECT_COMMIT: ''
  PROJECT_SUBMODULES: ''
  FETCH_SINGLE_BRANCH: true
SVN:
  PROJECT_URL: ''
  PROJECT_REVISION: ''
LOCAL:
  PROJECT_PATH: ''
PROJECT_FILES:
  FILE_PROJECT_CONF: ''
  FILE_JOBS_CONF: ''
  JOB_SCRIPTS_TYPE: ''
RERUN:
  RERUN: false
  RERUN_JOBLIST: ''
PLATFORMS:
  MARENOSTRUM4:
    TYPE: slurm
    HOST: mn1.bsc.es
    PROJECT: bsc32
    USER: bruno
    QUEUE: debug
    SCRATCH_DIR: /gpfs/scratch
    ADD_PROJECT_TO_HOST: false
    MAX_WALLCLOCK: 48:00
    TEMP_DIR: ''
    MAX_PROCESSORS: 128
JOBS:
  TRANSFER:
    PLATFORM: MARENOSTRUM4
    PROCESSORS: 1
    CHECK: on_submission
    FILE: test.sh
WRAPPERS:
  WRAPPER:
    TYPE: horizontal
    JOBS_IN_WRAPPER: TRANSFER

The key in the config above appear to be the CHECK: on_submission, and the WRAPPER.

To see the error message (hidden in the console by default, written to the log):

$ autosubmit create -cw a005 -np && autosubmit -lc DEBUG inspect a005
Checking configuration files...
Expdef config file is correct
Platforms sections: OK
Jobs sections OK
wrappers OK
Autosubmit general sections OK
Configuration files OK

The Experiment name is: a005
Sleep: 10
Loading JobList
Load finished
Creating jobs...
Creating TRANSFER jobs
Adding dependencies to the graph..
Changes detected, calculating dependencies
Adding dependencies for TRANSFER jobs
Adding dependencies to the job..
Transitive reduction...
Looking for edgeless jobs...
Loading JobList
Load finished
Creating jobs...
Creating TRANSFER jobs
Adding dependencies to the graph..
Changes detected, calculating dependencies
Adding dependencies for TRANSFER jobs
Adding dependencies to the job..
Transitive reduction...
Looking for edgeless jobs...
Length of the jobs list: 1
Sleep: 10
Starting to generate cmd scripts
Generating cmd scripts only for selected jobs
[WARNING] Generating the auxiliary job_list used for the -CW flag.
[WARNING] Aux Job_list was generated successfully
Loading parameters...
Parameters load.
Checking scripts...
1 of 1 checked
Scripts OK
Updating FAILED jobs
Updating WAITING jobs
Setting job: a005_TRANSFER status to: READY (all parents completed)...
Update finished
Loading parameters...
Parameters load.
Number of jobs available: 20
Number of jobs ready: 1
Jobs ready for MARENOSTRUM4: 1
Traceback (most recent call last):
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 2382, in submit_ready_jobs
    packages_to_submit = JobPackager(as_conf, platform, job_list, hold=hold).build_packages()
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/job/job_packager.py", line 468, in build_packages
    built_packages_tmp = self._build_horizontal_packages(jobs, wrapper_limits, section,wrapper_info=current_info)
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/job/job_packager.py", line 536, in _build_horizontal_packages
    package_jobs = horizontal_packager.build_horizontal_package()
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/job/job_packager.py", line 848, in build_horizontal_package
    if job.total_processors == "":
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/job/job.py", line 782, in total_processors
    return int(self.processors)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bdepaula/Development/python/workspace/autosubmit/bin/autosubmit", line 57, in main
    return_value = Autosubmit.parse_args()
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 693, in parse_args
    return Autosubmit.inspect(args.expid, args.list, args.filter_chunks, args.filter_status,
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 1516, in inspect
    Autosubmit.generate_scripts_andor_wrappers(
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 1607, in generate_scripts_andor_wrappers
    Autosubmit.submit_ready_jobs(as_conf, job_list, platforms_to_test, packages_persistence, True,
  File "/home/bdepaula/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 2417, in submit_ready_jobs
    raise AutosubmitCritical("This seems like a bug in the code, please contact AS developers", 7070, str(e))
log.log.AutosubmitCritical:  

Trace: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
 [CRITICAL] This seems like a bug in the code, please contact AS developers [eCode=7070]
More info at https://autosubmit.readthedocs.io/en/master/troubleshooting/error-codes.html

Will link with the existing merge request.

Edited Mar 12, 2024 by Bruno de Paula Kinoshita
Assignee
Assign to
Time tracking