Adjusting autosubmit parameters for the Genetic Algorithm (GA) parameter optimization workflow in AS 3.14.0
In this issue I plan to document all the adjustments needed in the Autosubmit workflow (specifically wrappers, processors per core, etc.) for optimal performance in the Genetic Algorithm (GA) parameter optimization experiments.
The workflow and typical experiments are described in this paper by Marcus Falls et al. 2022.
The autosubmit templates are in this repository.
In a typical GA experiment, an initial simulation (INI, template pre.sh
) is followed by a generation where 100 simulations (SIM, template sim.sh
) with different parameter sets run in parallel. Each of these simulations (individuals) is a CHUNK and runs on 1 core in less than 5 minutes. The outputs of each simulation are collected by the POST job (template post.sh
) to compute a cost function (based on different goodness-of-fit statistics) and select the refined parameter sets that will be used in the subsequent 100 generations. This process is repeated iteratively for N generations (= MEMBERS, typically no less than 40 and a maximum of 100) to obtain the optimal parameter set.
In this GA workflow developed by Marcus Falls, MEMBERS (= generations) are vertical and CHUNKS (= individual simulations with a given parameter set) are horizontal. In this issue Marcus documented his tests with horizontal-vertical wrappers in AS 3.13.0, which were not successful. Horizontal wrappers only did work in AS 3.13.0.
Now we need to run more experiments but adjustments to the wrappers and other AS parameters seem necessary, as we found when running a similar workflow used for sensitivity analyses documented in this GA issue and in this autosubmit issue.
I re-launched experiments a3tg (horizontal-vertical wrappers) and a41f (horizontal only) as a starting point for debugging. Both experiments had run smoothly in AS 3.13.0 in November 2021, although the addition of vertical in a3tg made no difference in practice.
Tagging @mcastril and @dbeltran , who already did debugging work for the sensitivity analyses. Keeping in the loop @rbernard , @macosta and @jllort. Thanks to all!