Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • autosubmit autosubmit
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 338
    • Issues 338
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 21
    • Merge requests 21
  • Deployments
    • Deployments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth SciencesEarth Sciences
  • autosubmitautosubmit
  • Issues
  • #748
Closed
Open
Issue created Oct 28, 2021 by Francesca Macchia@fmacchiaMaintainer

Job dependency on some members only

For the CAMS2-40 workflow, we need to run a synchronized job (DA) that only depend on some members. You can see an example of the workflow in a3wo, but a simplified version looks like this:

    graph TD;
      a3wo_20210802_000_1_SIM-->a3wo_20210802_1_DA
      a3wo_20210802_001_1_SIM-->a3wo_20210802_1_DA
      a3wo_20210802_002_1_SIM-->a3wo_20210802_1_DA
      a3wo_20210802_003_1_SIM-->a3wo_20210802_1_DA
      a3wo_20210802_000_1_SIM-->a3wo_20210802_003_REDUCE
      a3wo_20210802_001_1_SIM-->a3wo_20210802_003_REDUCE
      a3wo_20210802_002_1_SIM-->a3wo_20210802_003_REDUCE
      a3wo_20210802_003_1_SIM-->a3wo_20210802_003_REDUCE
      a3wo_20210802_1_DA-->a3wo_20210802_1_REDUCE_AN

with

[SIM]
RUNNING = chunk
[DA]
RUNNING = chunk
SYNCHRONIZE = member
[REDUCE]
RUNNING = member
FREQUENCY = 4
[REDUCE_AN]
RUNNING = chunk
SYNCHRONIZE = member

The idea is to have something like this:

    graph TD;
      a3wo_20210802_000_1_SIM-->a3wo_20210802_1_DA
      a3wo_20210802_001_1_SIM-->a3wo_20210802_1_DA
      a3wo_20210802_002_1_SIM-->a3wo_20210802_1_DA
      a3wo_20210802_003_1_SIM-->a3wo_20210802_003_1_REDUCE
      a3wo_20210802_1_DA-->a3wo_20210802_1_REDUCE_AN

so

  1. the DA job should depend only on 00[0-2]SIM
  2. the REDUCE job should run per chunk, only after x member (I need only reduce_003), and depending only on a3wo_20210802_003_1_SIM.

Could this be done in Autosubmit?

(I know, workflows with DA are always a challenge 😅 )

@dbeltran @wuruchi @mcastril

Assignee
Assign to
Time tracking