Job dependency on some members only
For the CAMS2-40 workflow, we need to run a synchronized job (DA) that only depend on some members. You can see an example of the workflow in a3wo, but a simplified version looks like this:
graph TD;
a3wo_20210802_000_1_SIM-->a3wo_20210802_1_DA
a3wo_20210802_001_1_SIM-->a3wo_20210802_1_DA
a3wo_20210802_002_1_SIM-->a3wo_20210802_1_DA
a3wo_20210802_003_1_SIM-->a3wo_20210802_1_DA
a3wo_20210802_000_1_SIM-->a3wo_20210802_003_REDUCE
a3wo_20210802_001_1_SIM-->a3wo_20210802_003_REDUCE
a3wo_20210802_002_1_SIM-->a3wo_20210802_003_REDUCE
a3wo_20210802_003_1_SIM-->a3wo_20210802_003_REDUCE
a3wo_20210802_1_DA-->a3wo_20210802_1_REDUCE_AN
with
[SIM]
RUNNING = chunk
[DA]
RUNNING = chunk
SYNCHRONIZE = member
[REDUCE]
RUNNING = member
FREQUENCY = 4
[REDUCE_AN]
RUNNING = chunk
SYNCHRONIZE = member
The idea is to have something like this:
graph TD;
a3wo_20210802_000_1_SIM-->a3wo_20210802_1_DA
a3wo_20210802_001_1_SIM-->a3wo_20210802_1_DA
a3wo_20210802_002_1_SIM-->a3wo_20210802_1_DA
a3wo_20210802_003_1_SIM-->a3wo_20210802_003_1_REDUCE
a3wo_20210802_1_DA-->a3wo_20210802_1_REDUCE_AN
so
- the DA job should depend only on 00[0-2]SIM
- the REDUCE job should run per chunk, only after x member (I need only reduce_003), and depending only on a3wo_20210802_003_1_SIM.
Could this be done in Autosubmit?
(I know, workflows with DA are always a challenge