This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
tools:autosubmit [2020/07/02 15:36] mcastril [Repository] |
tools:autosubmit [2020/08/18 16:05] mcastril |
||
---|---|---|---|
Line 14: | Line 14: | ||
The repository and issue tracker of Autosubmit is here | The repository and issue tracker of Autosubmit is here | ||
- | [[https:// | + | * [[https:// |
=== Useful links === | === Useful links === | ||
Line 20: | Line 20: | ||
* [[https:// | * [[https:// | ||
* [[http:// | * [[http:// | ||
- | * [[http:// | + | * [[http:// |
Line 27: | Line 27: | ||
[[http:// | [[http:// | ||
doi: 10.1109/ | doi: 10.1109/ | ||
- | |||
- | ==== Description ==== | ||
- | |||
- | |||
- | |||
- | === General Description === | ||
- | |||
- | |||
- | == Introduction == | ||
- | |||
- | A typical climate forecast experiment is a run of a climate model over a supercomputer having variable range of forecast length from a few months to a few years. And an experiment may have one or more than one start-dates and every start-date may comprise of single or many members. The full length of forecasting period for the experiment could be divided into number of chunks of fixed forecast length by exploiting the available options of model restart. Furthermore, | ||
- | {{file: | ||
- | |||
- | As we could see in the sample experiment which consists of 10 start-dates from 1960 to 2005 where every start-date is independent of each other and starting after every 5 years while each start-date comprise of 5 members. Every member is also independent and has been divided into 10 chunks which are dependent on each other. Now let us suppose that the forecast length for each chunk is one year and every chunk comprises of three types of jobs; a simulation (Sim), a post-processing (Post) and an archiving and cleaning job (Clean). Therefore with this typical exemplary experiment, one start-date with one member comprise of 30 jobs and eventually 1500 jobs will be run in total for the completion of the experiment. In short, there is a need of a system to automate such type of typical experiments and optimize the use of resources. | ||
- | |||
- | |||
- | == Goal == | ||
- | |||
- | Autosubmit is a tool to manage and monitor climate forecasting experiments by using supercomputers remotely and achieve the following goals: | ||
- | |||
- | * Efficient handling of highly dependent jobs | ||
- | * Optimum utilization of available computing resources | ||
- | * Ease of starting, stopping and live monitoring of experiments | ||
- | * Auto restarting the experiment or some part of experiment in case of failure | ||
- | * Use of database for experiment creation and assigning automatic experiment identity | ||
- | * Ability to reproduce the completed experiments fully or partially. | ||
- | {{file: | ||
- | |||
=== Technical Description === | === Technical Description === | ||
Line 159: | Line 131: | ||
You can check the style guide for Autosubmit [[tools: | You can check the style guide for Autosubmit [[tools: | ||
- | |||
- | ==== Update to the latest autosubmit development version using virtual environment ==== | ||
- | |||
- | Adapted from the wiki page about virtual environments | ||
- | [[ https:// | ||
- | |||
- | > ssh -X bscesautosubmit01 | ||
- | > module purge | ||
- | > mkdir -p ~/ | ||
- | > virtualenv ~/ | ||
- | > source ~/ | ||
- | > pip install https:// | ||
- | > autosubmit -v | ||
- | |||
- | In the meantime, if there has been autosubmit development, | ||
- | |||
- | > ssh -X bscesautosubmit01 | ||
- | > module purge | ||
- | > source ~/ | ||
- | > pip install --upgrade https:// | ||
- | > autosubmit -v | ||