diff --git a/autosubmit/job/job.py b/autosubmit/job/job.py index e5b921de281206d4d80cc4a06cfff80ae1e7f9d2..81f59e338a674b7ab36c56d02ddca8748be6dcab 100644 --- a/autosubmit/job/job.py +++ b/autosubmit/job/job.py @@ -1505,8 +1505,8 @@ class Job(object): """ Create the script content to be run for the job - :param as_conf: config - :type as_conf: config + :param as_conf: Autosubmit configuration object + :type as_conf: AutosubmitConfig :return: script code :rtype: str """ diff --git a/docs/source/devguide/fig3.png b/docs/source/devguide/fig3.png deleted file mode 100644 index 9a330fd50118a8f278cf8af115aea6448ee7df83..0000000000000000000000000000000000000000 Binary files a/docs/source/devguide/fig3.png and /dev/null differ diff --git a/docs/source/devguide/index.rst b/docs/source/devguide/index.rst deleted file mode 100644 index a086bb867e356d35e90bb10d0d6edf5babb3500f..0000000000000000000000000000000000000000 --- a/docs/source/devguide/index.rst +++ /dev/null @@ -1,39 +0,0 @@ -############################## -Developing an EC-Earth Project -############################## - -Autosubmit is used at BSC to run EC-Earth. To do that, a git repository has been created that contains the model source -code and the scripts used to run the tasks. - -.. figure:: fig3.png - :width: 70% - :align: center - :alt: EC-Earth experiment - - Example of monitoring plot for EC-Earth run with Autosubmit for 1 start date, 1 member and 3 chunks. - -The workflow is defined using seven job types, as shown in the figure above. These job types are: - -- Local_setup: prepares a patch for model changes and copies it to HPC. -- Remote_setup: creates a model copy and applies the patch to it. -- Ini: prepares model to start the simulation of one member. -- Sim: runs a simulation chunk (usually 1 to 3 months). -- Post: post-process outputs for one simulation chunk. -- Clean: removes unnecessary outputs from the simulated chunk. -- Transfer: transfers post-processed outputs to definitive storage. - -Autosubmit can download the project from git, svn and local repositories via the parameter `PROJECT.PROJECT_TYPE`. When the source is a git one, the user can specify the submodules, commit, branch, and tag. - -In addition, the user can also alter the git behaviour and specify other optimization parameters such as: - - Fetching one single branch - - Depth of the submodules. - - -The different projects contain the shell script to run, for each job type (local setup, remote setup, ini, sim, post, clean and transfer) that are platform independent. -Additionally the user can modify the sources under proj folder. -The executable scripts are created at runtime so the modifications on the sources can be done on the fly. - -.. warning:: Autosubmit automatically adds small shell script code blocks in the header and the tailer of your scripts, to control the workflow. - Please, remove any exit command in the end of your scripts, e.g. ``exit 0``. - -.. important:: For a complete reference on how to develop an EC-Earth project, please have a look in the following wiki page: https://earth.bsc.es/wiki/doku.php?id=models:models \ No newline at end of file diff --git a/docs/source/index.rst b/docs/source/index.rst index 06a425cfde0e08c7741a09480bcbeca846ac7db9..37b944a0766855de97b43ef4b64db0046569f7f8 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -44,6 +44,7 @@ Welcome to autosubmit's documentation! /userguide/set_and_share_the_configuration/index /userguide/variables /userguide/expids + /userguide/templates .. toctree:: :caption: Database Documentation @@ -52,13 +53,6 @@ Welcome to autosubmit's documentation! /database/index -.. toctree:: - :caption: Developer Guide - :maxdepth: 1 - :hidden: - - /devguide/index - .. toctree:: :caption: Troubleshooting :maxdepth: 1 diff --git a/docs/source/userguide/monitor_and_check/index.rst b/docs/source/userguide/monitor_and_check/index.rst index 4dbee3148845e64eef1a9e623beda03392bef8e1..a407c787d18f02fc73811945f3f3d87e1a3da587 100644 --- a/docs/source/userguide/monitor_and_check/index.rst +++ b/docs/source/userguide/monitor_and_check/index.rst @@ -81,6 +81,8 @@ For example: SHOW_CHECK_WARNINGS: FALSE ... +.. _inspect_cmd: + How to generate cmd files ------------------------- diff --git a/docs/source/userguide/templates.rst b/docs/source/userguide/templates.rst new file mode 100644 index 0000000000000000000000000000000000000000..e8a624ae4db1cffb07698b3225c3b8f1a8a7751f --- /dev/null +++ b/docs/source/userguide/templates.rst @@ -0,0 +1,88 @@ +################ +Script templates +################ + +Autosubmit jobs require a ``FILE`` property that points to a +script template. Script templates can be written in Bash shell, +R, or Python. By default, the ``TYPE`` property of a job is set +to ``bash``. Template scripts can have any file extension, +the generated script will have it replaced by ``.cmd``. + +.. code-block:: yaml + :emphasize-lines: 3 + :caption: Job ``JOB_1`` with template script ``print_expid.sh`` + + JOBS: + JOB_1: + FILE: print_expid.sh + PLATFORM: LOCAL + RUNNING: once + TYPE: bash # default + +The script template ``print_expid.sh`` file must exist in the +Autosubmit Project. When you run ``autosubmit create`` or +``autosubmit refresh``, Autosubmit will copy the Project files, +including template scripts, to the experiment folder `proj`. +The template scripts are then preprocessed and the final script +is generated when an Autosubmit experiment is +:ref:`inspected ` or :doc:`created `. + +When Autosubmit preprocesses the template script, it replaces +*placeholders* by configuration values. A placeholder is any +configuration key wrapped by ``%%``. For example, ``%DEFAULT.EXPID%`` +shown below refers to the Autosubmit configuration value +found in the ``DEFAULT.EXPID`` YAML configuration key. + +Assuming that one of the Autosubmit experiment configuration files +contains the following: + +.. code-block:: yaml + :caption: Autosubmit configuration + + DEFAULT: + EXPID: a000 + # ... other settings + +and a template script used in that experiment contains the following +code: + +.. code-block:: bash + :caption: A template script + + #!/bin/bash + + echo "The experiment ID is %DEFAULT.EXPID%" + +then after the template script is preprocessed by Autosubmit, the +generated script will be written in the experiment folder with the +extension ``.cmd`` and it will look like this: + +.. code-block:: bash + :caption: Generated script + + #!/bin/bash + + # Autosubmit Header + # ... + + echo "The experiment ID is a000" + + # Autosubmit Tailer + +Header and Tailer +================= + +Autosubmit does not require users to modify the header and +tailer used. The header contains code to set the correct locale, +create supporting files for Autosubmit (e.g. ``_TEST_STAT``), +and functions used internally by Autosubmit (e.g. for checkpointing). + +The tailer creates other supporting files (e.g. ``_TEST_COMPLETED`` +and update ``_TEST_STAT``). + +If necessary, Autosubmit allows users to customize the header +and the tailer. The configuration keys ``EXTENDED_HEADER_PATH`` and +``EXTENDED_TAILER_PATH`` can be used to indicate the location +of, respectively, the header and tailer scripts used by Autosubmit. +The location is relative to the project root folder, and the scripts +are appended after the default header and tailer scripts.