|
The way to use Autosubmit in the Earth Sciences network is by loading the corresponding module (module load autosubmit). If none version is specified, the command will load the **default version**, which is always the **preferred** one. For a version to become the default, it has to pass a test period of some months during which the software is tested, both in the Auto-models' weekly tests and in production experiments.
|
|
The way to use Autosubmit in the Earth Sciences network is by loading the corresponding module (module load autosubmit). If none version is specified, the command will load the **default version**, which is always the **preferred** one. For a version to become the default, it has to pass a test period of some months during which the software is tested, both in the Auto-models' weekly tests and in production experiments.
|
|
|
|
|
|
This means that the **default** version is **proved** to be robust and it is the recommender option for production experiments. However, some features and improvements are only available in development versions, that we recommend to test so they can be improved further and become operational, always under the **user's responsibility**.
|
|
This means that the **default** version is **proved** to be robust and it is the recommender option for production experiments. However, some features and improvements are only available in development versions, which we recommend testing so they can be improved further and become operational, always under the **user's responsibility**.
|
|
|
|
|
|
The aim of this section is to provide useful **guidelines** about the features that the default and development versions provide, as well as recommendations to run experiment configurations that may not be fully supported at the moment:
|
|
The aim of this section is to provide useful **guidelines** about the features that the default and development versions provide, as well as recommendations to run experiment configurations that may not be fully supported at the moment:
|
|
|
|
|
|
|
|
|
|
* **3.12.0** - This is version is already outdated, and users should run 3.13.0.
|
|
* **3.12.0** - This is version is already outdated, and users should run 3.13.0.
|
|
|
|
|
|
|
|
|
|
* In this version **vertical** and **horizontal** **wrappers** are fully supported.
|
|
* In this version **vertical** and **horizontal** **wrappers** are fully supported.
|
|
* **Horizontal-vertical** wrappers are supported too. They were first developed in this Autosubmit version and they have been used in production together.
|
|
* **Horizontal-vertical** wrappers are supported too. They were first developed in this Autosubmit version and they have been used in production together.
|
|
* Due to **technical limitations**, we don't recommend running experiments having many startdates/members (increased concurrency) or very large wrappers with 3.12.0b. As a rule of thumb, experiments with more than 10-20 members in total or wrappers with more than 50 jobs (the user can always reduce the wrapper size) may experience delays in the Autosubmit refresh cycle and generating the monitor views.
|
|
* Due to **technical limitations**, we don't recommend running experiments having many startdates/members (increased concurrency) or very large wrappers with 3.12.0b. As a rule of thumb, experiments with more than 10-20 members in total or wrappers with more than 50 jobs (the user can always reduce the wrapper size) may experience delays in the Autosubmit refresh cycle and generating the monitor views.
|
... | @@ -15,8 +14,6 @@ The aim of this section is to provide useful **guidelines** about the features t |
... | @@ -15,8 +14,6 @@ The aim of this section is to provide useful **guidelines** about the features t |
|
* Autosubmit **migrate** is not secured for big experiments, so it is recommended to backup the offered experiment (in the remote platforms) first. We encourage to use only 3.13.0 migrate.
|
|
* Autosubmit **migrate** is not secured for big experiments, so it is recommended to backup the offered experiment (in the remote platforms) first. We encourage to use only 3.13.0 migrate.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* **3.13.0** - This is the actual default version and provides a lot of improvements, especially in terms of efficiency and stability. It also brings a full refactor of the wrapper module.
|
|
* **3.13.0** - This is the actual default version and provides a lot of improvements, especially in terms of efficiency and stability. It also brings a full refactor of the wrapper module.
|
|
|
|
|
|
* **Multi-threaded** wrappers were introduced in this version.
|
|
* **Multi-threaded** wrappers were introduced in this version.
|
... | @@ -44,10 +41,39 @@ The aim of this section is to provide useful **guidelines** about the features t |
... | @@ -44,10 +41,39 @@ The aim of this section is to provide useful **guidelines** about the features t |
|
* Reworked **migrate** command, with improvements in robustness and security.
|
|
* Reworked **migrate** command, with improvements in robustness and security.
|
|
* New `pklfix` command to restore a corrupted local database.
|
|
* New `pklfix` command to restore a corrupted local database.
|
|
* New `updatedescrip` command to modify the experiment's description.
|
|
* New `updatedescrip` command to modify the experiment's description.
|
|
|
|
*
|
|
|
|
|
|
* **3.14.0b** - This is the development version and provides new functionalities especially in terms of workflow flexibility.
|
|
* **3.14.0b** - This is the development version and provides new functionalities especially in terms of workflow flexibility.
|
|
|
|
|
|
* (...)
|
|
* Workflows have even more **flexibility** than in previous iterations:
|
|
|
|
* Improved select_chunks to be less restrictive and be able to use along with other parameters.
|
|
|
|
* Added select_members, which it functionality is the similar to select_chunk but for members.
|
|
|
|
* Added conditional dependencies, now AS is able to work with conditional dependencies in the manner that a dependency can be marked as optional ('?') char.
|
|
|
|
* Rerun has been reintroduced, it comes with improvements to the job selection. Now it allows to rerun any kind of job.
|
|
|
|
* Improviments to wrappers:
|
|
|
|
* Added the possibility of run multiple wrappers under the same experiment.
|
|
|
|
* Added an internal retrial mechanism for vertical wrappers, this bypasses the resubmission of the wrapper.
|
|
|
|
* Added an extensible wallclock in cases where the internal retrial mechanism is triggered.
|
|
|
|
* Added the possibility of increase the time for perform a job retrial.
|
|
|
|
* Added X11 support.
|
|
|
|
* Now LANG is enforced to use the same across all autosubmit iterations ( wrapper,code,runtime and experiments).
|
|
|
|
* Now proj.conf supports %_% variable nomenclature.
|
|
|
|
* Added security to autosubmit folders, having only tmp accessible for other users.
|
|
|
|
* Improved the job_status log and added a new file for failed jobs.
|
|
|
|
* Autosubmit stats has been reworked and improved.
|
|
|
|
* Now non-owner users for a given experiment are able to monitor others users experiments without activate anykind of change into the experiment folder ( except for tmp folder).
|
|
|
|
* Redesigned Allowed/Denied hosts. Now is possible to cherry-pick exactly which machines are able to launch autosubmit and which command.
|
|
|
|
* Now Autosubmit users will receive an e-mail if a remote platform has issues. It will stated which platform has the issue.
|
|
|
|
* Added security and a cancel mechanism into auto submit recovery, if -f flag is not specified recovery won't do anything and will warn that there are active jobs.
|
|
|
|
* Added an chunk dependant wallclock time.
|
|
|
|
* Added an IO exception recovery.
|
|
|
|
* And finally, some fixes were added into 3.13.0 and 3.14.0b.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
... | | ... | |