3.13.0 - This has been the Autosubmit development versions for many months, so it provides a lot of improvements, specially in terms of efficiency and stability. It also brings a full refactor of the wrappers module. Multi-threaded wrappers were introduced in this version. It provides the possibility to specify multiple hosts for the same platform (in a list) so it is more robust against connection issues / login failures. In general, big experiments, with many startdates/members or featuring very big wrappers run much more efficiently with 3.13.0. A completely new implementation of remote dependencies (PreSubmission) were introduced in this version. It helps to speed up the jobs in a Slurm platform by sending the next 10 Waiting jobs in advance to the queues. Workflows have more flexibility by the inclusion of a new way to define dependencies for specific chunks. Changes were made to the algorithm that handles the maximum active jobs by platform. From this version, wrapped jobs count as a single job for Autosubmit, and the maximum number of inner jobs can be defined with new wrapper parameters. There is a new log (.err, .out, COMPLETED, STAT files) recover system, that performs re-tries (in background threads) of the log files transfer from the remote platforms in case of failure. The user can specify a datetime or time when the experiment should start by sending the -st flag (plus the right format) using the autosubmit run command. The user can specify an experiment dependency by sending the -sa (plus the right expid format) using the autosubmit run command. The experiment will start when the experiment specified in the -sa flag finishes. When the user quits Autosubmit by using the CTRL+C keys, Autosubmit will make sure all threads are finished correctly before closing. Job lifecycle information is stored in an external database that will allow the user to see the historical information of the execution of the jobs in her experiment. This information is gathered in a way that does not interfere with the normal workflow, even if the information gathering fails or any of its components. Furthermore, threading is implemented to prevent unnecessary delays. Specific members can now be configured to run by using the autosubmit run together with the -rm command. By setting this flag, Autosubmit will only run jobs belonging to the supplied members. Jobs that were running before this command was sent, will be monitored and properly completed. The git clone operation now implements a backup procedure that will prevent loss of information in case of wrong configuration or network error. There is an improvement of the security, now all commands that could change the workflow are locked by an owner-only mechanism. Ej: create,refresh and run. Added autosubmit dbfix expid command that allows users to fix the database malformed error. Custom shebang (header of the script templates) so it is possible to use Python or R templates with a specific Python/R version dependency, for example. Workflow updates: A difference of 3.12.0 version, in this one, only Create and run commands can regenerate the configuration, workflow and structure information. In case of run, only will be updated if a change is detected before the starting of the main run loop. Improves into failure tolerance, in this version AS will try to prevent as many errors as possible at the beginning of the run and other delicate operations, instead of detecting them while in running time. Allows prioritizing a list of jobs to be run before the rest of the workflow. Via the use of the Two_step_start variable set in expdef.conf Allows skipping jobs under the same section if their last on queuing member/chunks is higher than other on queuing/waiting/ready 3.12.0 - Shortly, this will be the Autosubmit default. Therefore, it is the recommended software to run any kind of experiments unless there is a necessity only covered by the development versions. In this version vertical and horizontal wrappers are fully supported. Horizontal-vertical wrappers are supported too. They were first developed in this Autosubmit version and they have been used in production together. Due to technical limitations, we don't recommend to run experiments having many startdates/members (increased concurrency) or very large wrappers with 3.12.0b. As a rule of thumb, experiments with more than 10-20 members in total or wrappers with more than 50 jobs (the user can always reduce the wrapper size) may experience delays in the Autosubmit refresh cycle and generating the monitor views. In this version, inner jobs inside QUEUING wrappers show a SUBMITTED status. This is fixed in 3.12.1b. 3.11.1 Fix minor issues Added new command for describe experiment 3.11.0 Included %m% in the list of exceptions Wrapper major refactoring - WrapperJob, WrapperBuilder and WrapperFactory - 2 types of hybrid wrapper: vertical-horizontal and horizontal-vertical - Machinefiles for horizontal and hybrid - Reduced submitting time by merging commands (rm and find) into one Fixed stats plot for wrapped jobs Checks for necessary configuration when defining wrapper in autosubmit.conf: MAX_WALLCLOCK, MAX_PROCESSORS, PROCESSORS_PER_NODE Wrapper regression tests for mn4 Flag option not to do transitive reduction Added the reasons for QUEUING (Reason) as returned by SLURM Added master as default branch for git projects (otherwise clone failed if empty) Some bug fixes: - Bug fix for changing status of synchronized jobs - Bug fix related to grouping - Bug fix related to variable substitution %% 3.10.0 Vertical wrapper allowing mixed job types and additional constraints Job grouping in the visualization graph Txt output status for autosubmit monitor Setstatus with allowing multiple job types Host whitelist option in .autosubmitrc for autosubmit run DELAY and SPLITS options for job configuration Setting blank value for absent variables in project configuration Minor bug fixes 3.9.0 Custom directives for the HPC resource manager headers can be added on platforms and jobs configuration files ~ only paramiko (LSF, SLURM and PBS) First version with migrate experiments (to another user) On CCA, TASKS and THREADS can be expressed in lots (e.g. 127:1) Some bug fixes: - QUEUE on slurm specified on directive qos instead of partition - Variable expansion on CCA (ECMWF) headers 3.8.1 First version with job packages ~ only paramiko (LSF, SLURM and PBS) - Vertical - Horizontal - With dependencies ~ only for vertical Python wrapper for CCA (ECMWF) On submission template checking Some UX improvements Other minor bug fixes 3.8.0 First version with LSF arrays: - Include all the bug fixes & features from 3.7.7 - NOT include the bug fixes from 3.7.8 3.7.8 Some bug fixes: - Database persistence - Delete command - Unarchive command - CHUNKINI option - Paramiko permissions - Paramiko non-existing remote copy 3.7.7 Some improvements for Slurm platforms Geo-definition of processors New configuration variables: - CHUNKINI - MEMORY - MEMORY_PER_TASK - HYPERTHREADING Other minor bug fixes 3.7.6 Fixed refresh Fixed recovery for ECMWF Local logs copy can be disabled Some UX improvements Other minor bug fixes 3.7.5 Fixed minor with LSF's logs 3.7.4 Forward dependencies Performance improvements Log files copied into LOCAL platform PROCESSORS_PER_NODE/TASKS now optional Exclusivity for MN3 (with Paramiko) THREADS optional for ECMWF Minor bug fixes 3.7.3 Fixed error with logs directives (err & out were swapped) Added new option for MN3: SCRATCH_FREE_SPACE PROCESSORS_PER_NODE/TASKS now available with Paramiko Other minor bug fixes 3.7.2 Minor bug fixes Regression test suite improved Solved some problems with paramiko & ECMWF platform 3.7.1 Fixed issue in setstatus Added new 'testcase' command 3.7.0 Big improvements on memory consumption Added new configuration variables (default job's type, number of members..) Added an alternative method to configure autosubmit without dialog library UX improved (logs fixed, exceptions handled) Fixed error with COMPLETED jobs shown as FAILED Fixed error with LSF schedulers by default Fixed bug on stats feature Fixed some bugs with Git and SVN Other minor bug fixes 3.6.1 Fixed an incompatibility with recent versions of radical.utils (saga) 3.6.0 Added multi-library communications support: SAGA & Paramiko UX improved on some error cases Fixed permission backwards incompatibility Fixed authorization problems on SAGA implementation Other minor bug fixes 3.5.0 Added another mechanism for SAGA errors prevention Added no-plot option to setstatus Added exclusivity and processes per host support for MN Check method fixed (not working since 3.2) Other minor bug fixes 3.4.1 Hot-fix ECMWF binary (bash, R, python) Hot-fix Mail Notifications 3.4.0 Added email notifications support Added mechanisms for incoherence prevention Added mechanisms for SAGA pty errors prevention 3.3.1 Fixed bug with no-leap experiments 3.3.0 Added filters in monitor Added support for Python jobs Added support for R jobs Added unitary test suite Synchronize job param Fixed recovery issue Other minor bugs fixed 3.2.0 Changed WAIT default Recovery without -s Group permissions to log files Reservation support for MN Fixed retrials bug Fixed rerun bug Fixed stats bug Other minor bugs fixed 3.2.0b3 SAGA related bug fixes Minor bug fixes 3.2.0b2 Minor bug fixes 3.2.0b1 Now using SAGA for connection and queue management, adding support for more queue types Stats revamped to provide more information and make it available earlier. 3.1.9 Hot fix LOCAL platform 3.1.8 Hot fix LOCAL platform 3.1.7 Fix issue StatsSnippet (job_st typo) Fix issue recovery platforms to test Fix issue s_rt SGE directive 3.1.6 Fix issue when creating, no option FILE_JOBS_CONF Fixes in documentation 3.1.5 Connect fixed to use Proxy Command Fixes in documentation 3.1.4 Documentation for Variables Minor bug fixes 3.1.3 Minor bug fixes, mostly related to SLURM 3.1.2 Minor bug fixes 3.1.1 Fix for issue with 'noleap' calendar 3.1.0 Added archive and unarchive commands 3.0.6 Fixed bug in setstatus. Change in test. 3.0.5 Fixed bug in recovery. 3.0.4 Fixed bug in platform headers. Fixed bug in delete. Added readme and changelog commands. MAX_WAITING_JOBS and TOTAL_JOBS now defined by platform. Simplified console output of run sub command. 3.0.3 Fixed bug in expid test. 3.0.2 Fixed bug in the local platform. 3.0.1 Fixed bug in config. 3.0.0 Restructure layout.