... | @@ -4,15 +4,12 @@ This means that the **default** version is **proved** to be robust and it is the |
... | @@ -4,15 +4,12 @@ This means that the **default** version is **proved** to be robust and it is the |
|
|
|
|
|
The aim of this section is to provide useful **guidelines** about the features that the default and development versions provide, as well as recommendations to run experiment configurations that may not be fully supported at the moment:
|
|
The aim of this section is to provide useful **guidelines** about the features that the default and development versions provide, as well as recommendations to run experiment configurations that may not be fully supported at the moment:
|
|
|
|
|
|
|
|
|
|
* **3.12.0** - This is version is already outdated, and users should run 3.13.0.
|
|
* **3.12.0** - This is version is already outdated, and users should run 3.13.0.
|
|
* In this version **vertical** and **horizontal** **wrappers** are fully supported.
|
|
* In this version **vertical** and **horizontal** **wrappers** are fully supported.
|
|
* **Horizontal-vertical** wrappers are supported too. They were first developed in this Autosubmit version and they have been used in production together.
|
|
* **Horizontal-vertical** wrappers are supported too. They were first developed in this Autosubmit version and they have been used in production together.
|
|
* Due to **technical limitations**, we don't recommend running experiments having many startdates/members (increased concurrency) or very large wrappers with 3.12.0b. As a rule of thumb, experiments with more than 10-20 members in total or wrappers with more than 50 jobs (the user can always reduce the wrapper size) may experience delays in the Autosubmit refresh cycle and generating the monitor views.
|
|
* Due to **technical limitations**, we don't recommend running experiments having many startdates/members (increased concurrency) or very large wrappers with 3.12.0b. As a rule of thumb, experiments with more than 10-20 members in total or wrappers with more than 50 jobs (the user can always reduce the wrapper size) may experience delays in the Autosubmit refresh cycle and generating the monitor views.
|
|
* In this version, inner jobs inside QUEUING wrappers show a SUBMITTED status. This is fixed in 3.13.0.
|
|
* In this version, inner jobs inside QUEUING wrappers show a SUBMITTED status. This is fixed in 3.13.0.
|
|
* Autosubmit **migrate** is not secured for big experiments, so it is recommended to backup the offered experiment (in the remote platforms) first. We encourage to use only 3.13.0 migrate.
|
|
* Autosubmit **migrate** is not secured for big experiments, so it is recommended to backup the offered experiment (in the remote platforms) first. We encourage to use only 3.13.0 migrate.
|
|
|
|
|
|
|
|
|
|
* **3.13.0** - This is the actual default version and provides a lot of improvements, especially in terms of efficiency and stability. It also brings a full refactor of the wrapper module.
|
|
* **3.13.0** - This is the actual default version and provides a lot of improvements, especially in terms of efficiency and stability. It also brings a full refactor of the wrapper module.
|
|
* **Multi-threaded** wrappers were introduced in this version.
|
|
* **Multi-threaded** wrappers were introduced in this version.
|
|
* It provides the possibility to specify **multiple hosts** for the same platform (in a list) so it is more robust against connection issues/login failures.
|
|
* It provides the possibility to specify **multiple hosts** for the same platform (in a list) so it is more robust against connection issues/login failures.
|
... | @@ -40,53 +37,50 @@ The aim of this section is to provide useful **guidelines** about the features t |
... | @@ -40,53 +37,50 @@ The aim of this section is to provide useful **guidelines** about the features t |
|
* New `pklfix` command to restore a corrupted local database.
|
|
* New `pklfix` command to restore a corrupted local database.
|
|
* New `updatedescrip` command to modify the experiment's description.
|
|
* New `updatedescrip` command to modify the experiment's description.
|
|
* Added Nord3 support.
|
|
* Added Nord3 support.
|
|
|
|
* **<span dir="">3.14.0</span>**<span dir=""> - This latest development version provides new functionalities, especially regarding workflow flexibility.</span>
|
|
* **3.14.0** - This latest development version for autosubmit 3 provides new functionalities, especially regarding workflow flexibility.
|
|
* **<span dir="">Workflows have now increased flexibility which includes:</span>**
|
|
|
|
* <span dir="">There were improvements in **select_chunks** and added **select_members.**</span>
|
|
* Workflows have now increased flexibility which includes:
|
|
* <span dir="">A new way of setting up the dependencies in the form of weak dependencies when specified in a job allows some dependencies to fail (dependencies marked with the '?' char).</span>
|
|
* There were improvements in select_chunks and added select_members.
|
|
* <span dir="">Added an **Exclusion** parameter that allows disabling the creation of a job for a given member/chunk.</span>
|
|
* A new way of setting up the dependencies in the form of weak dependencies when specified in a job allows some dependencies to fail (dependencies marked with the '?' char).
|
|
* <span dir="">Reintroduced the **re-run mechanism** with improvements to the job selection and now it allows re-running any job.</span>
|
|
* Added an Exclusion parameter that allows disabling the creation of a job for a given member/chunk.
|
|
* **<span dir="">Workflow behaviour configuration:</span>**
|
|
* Reintroduced the re-run mechanism with improvements to the job selection and now it allows re-running any job.
|
|
* <span dir="">Added an **extensible wall clock** in cases where the internal retrial mechanism is triggered.</span>
|
|
* Workflow behaviour configuration:
|
|
* <span dir="">Added the possibility of **delaying job retrials.**</span>
|
|
* Added an extensible wall clock in cases where the internal retrial mechanism is triggered.
|
|
* <span dir="">Now **Proj.conf** supports dynamic variables (%\_%).</span>
|
|
* Added the possibility of delaying job retrials.
|
|
* <span dir="">We Added a chunk-dependant **wallclock** time.</span>
|
|
* Now Proj.conf supports dynamic variables (%_%).
|
|
* <span dir="">We standardized the meaning of task, total_jobs, and threads across all platforms and added parameter tasks to the slurm scheduler and hyperthreading for cca. </span>
|
|
* We Added a chunk-dependant wallclock time.
|
|
* **<span dir="">Improvements to wrappers:</span>**
|
|
* We standardized the meaning of task, total_jobs, and threads across all platforms and added parameter tasks to the slurm scheduler and hyperthreading for cca.
|
|
* <span dir="">Added the possibility of running **different types of wrappers** under the same experiment.</span>
|
|
* Improvements to wrappers:
|
|
* <span dir="">Added an **internal retrial** mechanism for vertical wrappers, bypassing the resubmission of the wrapper.</span>
|
|
* Added the possibility of running different types of wrappers under the same experiment.
|
|
* **<span dir="">Experiment stats improvements:</span>**
|
|
* Added an internal retrial mechanism for vertical wrappers, bypassing the resubmission of the wrapper.
|
|
* <span dir="">Job historical database is improved.</span>
|
|
* Experiment stats improvements:
|
|
* <span dir="">Redesigned **dbfix** command to restore the historical job database in case of corruption. </span>
|
|
* Job historical database is improved.
|
|
* <span dir="">Now, AS can detect and recover the corruption of the historical job database in real time.</span>
|
|
* Redesigned dbfix command to restore the historical job database in case of corruption.
|
|
* <span dir="">The autosubmit **stats** command is more detailed.</span>
|
|
* Now, AS can detect and recover the corruption of the historical job database in real time.
|
|
* <span dir="">Homogeneous **locale (LANG)** enforced across all autosubmit iterations (wrapper, code, runtime and experiments).</span>
|
|
* The autosubmit stats command is more detailed.
|
|
* **<span dir="">Project data/code management</span>**
|
|
* Homogeneous locale (LANG) enforced across all autosubmit iterations (wrapper, code, runtime and experiments).
|
|
* <span dir="">Depth and githooks are now supported.</span>
|
|
* Project data/code management
|
|
* <span dir="">Submodules of submodules are now supported.</span>
|
|
* Depth and githooks are now supported.
|
|
* <span dir="">Better project transfer when used in local mode. </span>
|
|
* Submodules of submodules are now supported.
|
|
* **<span dir="">Security changes</span>**
|
|
* Better project transfer when used in local mode.
|
|
* <span dir="">Security increased based on owner-only writing permissions for Autosubmit folders, having only tmp accessible for other users.</span>
|
|
* Security changes
|
|
* <span dir="">Redesigned **Allowed/Denied** hosts: authorized/forbidden mechanism for specific hosts and commands.</span>
|
|
* Security increased based on owner-only writing permissions for Autosubmit folders, having * only tmp accessible for other users.
|
|
* <span dir="">Users who do not own a given experiment can **monitor** others' experiments without changing the experiment tree (except the temp folder).</span>
|
|
* Redesigned Allowed/Denied hosts: authorized/forbidden mechanism for specific hosts and commands.
|
|
* <span dir="">We Added security and a cancel mechanism into Autosubmit **recovery** (it will detect existing active jobs).</span>
|
|
* Users who do not own a given experiment can monitor others' experiments without changing the experiment tree (except the temp folder).
|
|
* **<span dir="">Notifications changes:</span>**
|
|
* We Added security and a cancel mechanism into Autosubmit recovery (it will detect existing active jobs).
|
|
* <span dir="">Now Autosubmit users will receive an **e-mail notification** if there are issues with any remote platform.</span>
|
|
* Notifications changes:
|
|
* <span dir="">Now Autosubmit users will always receive their e-mails with the proper timestamp.</span>
|
|
* Now Autosubmit users will receive an e-mail notification if there are issues with any remote platform.
|
|
* **<span dir="">Improvements across all recovery procedures.</span>**
|
|
* Now Autosubmit users will always receive their e-mails with the proper timestamp.
|
|
* <span dir="">New I/O recovery.</span>
|
|
* Improvements across all recovery procedures.
|
|
* <span dir="">New scheduler errors recovery.</span>
|
|
* New I/O recovery.
|
|
* <span dir="">Log retrieval features a better recovery mechanism.</span>
|
|
* New scheduler errors recovery.
|
|
* **<span dir="">Improved expid</span>**<span dir=""> commands regarding the **expid deletion, database storage and messages provided.**</span>
|
|
* Log retrieval features a better recovery mechanism.
|
|
* **<span dir="">The improved migrate command</span>**<span dir=""> now supports bigger file sizes and an increment in the transfer speed.</span>
|
|
* Improved expid commands regarding the expid deletion, database storage and messages provided.
|
|
* <span dir="">New **installation methods** are supported ( pip, conda )</span>
|
|
* The improved migrate command now supports bigger file sizes and an increment in the transfer speed.
|
|
* **<span dir="">Docs</span>**<span dir=""> structure and content are improved. </span>
|
|
* New installation methods are supported ( pip, conda )
|
|
* <span dir="">Implements a set of **new optimizations** that address some issues regarding memory and performance. </span>
|
|
* Docs structure and content are improved.
|
|
* **<span dir="">4.0.0b</span>**<span dir=""> - This latest beta version complete overhaul of the configuration management alongside the porting to python 3.7 and 3.9.</span>
|
|
* Implements a set of new optimizations that address some issues regarding memory and performance.
|
|
* <span dir="">Written in Python 3. </span>
|
|
|
|
* TODO
|
|
* **4.0.0b** - This latest beta version features a complete overhaul of the configuration management alongside the porting to python 3.7 and 3.9.
|
|
|
|
* Written in Python 3.
|
|
|
|
* TODO
|
|
|
|
|
|
|
|
Further information about all these functionalities and commands can be found at the [Autosubmit documentation](https://autosubmit.readthedocs.io/en/latest/introduction.html). |
|
Further information about all these functionalities and commands can be found at the [Autosubmit documentation](https://autosubmit.readthedocs.io/en/latest/introduction.html). |
|
|
|
\ No newline at end of file |