This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
tools:run_1st_exp [2016/05/31 13:36] mmenegoz |
tools:run_1st_exp [2018/11/27 16:57] rwhite |
||
---|---|---|---|
Line 3: | Line 3: | ||
The idea of this documentation is to provide information for people running climate models at BSC. The goal is not to replace the autosubmit documentation, | The idea of this documentation is to provide information for people running climate models at BSC. The goal is not to replace the autosubmit documentation, | ||
- | ** Please, feel free to fill this documentation ** | + | ** Please, feel free to fill and/or correct |
- | Documentation to be checked and completed | + | Documentation to be checked and completed. It should be good that people include in this page some **synthesised** information about how to debug climate models at BSC. The part concerning the post-processing of experiment also need to be completed. \\ |
- | It should be good that people include in this page some **synthesised** information about how to debug climate models at BSC. The part concerning the post-processing of experiment also need to be completed. \\ | + | Modification History: \\ |
+ | Martin Ménégoz, 31/05/2016 \\ | ||
+ | Simon Wild, 09/04/2018 \\ | ||
+ | |||
+ | This user guide below is rather obsolete - it contains some helpful overview information and is worth reading through. For a working tutorial and an updated userguide you should look here: [[https:// | ||
+ | and here [[https:// | ||
** First step: Use autosubmit** | ** First step: Use autosubmit** | ||
- | To launch an experiment at BSC, for example a simulation with the ocean-atmosphere coupled version of EC-Earth, we have at our disposal **autosubmit**, | + | To launch an experiment at BSC, for example a simulation with the ocean-atmosphere coupled version of EC-Earth, we have at our disposal **autosubmit**, |
+ | In the autosubmit tutorial under the above link some updates are necessary: HPC name should be changed to ' | ||
+ | the gitlab url needs to be updated to ' | ||
+ | |||
+ | To load autosubmit in Earth infrastructure, | ||
+ | module load autosubmit | ||
NB1: the information needed by autosubmit to prepare an experiment (model version, HPC, etc...) is set up in the file expdef_${exp}.conf. In particular, you have to indicate which model sources you want to use. At BSC, each model version appears generally under a git project: for example: https:// | NB1: the information needed by autosubmit to prepare an experiment (model version, HPC, etc...) is set up in the file expdef_${exp}.conf. In particular, you have to indicate which model sources you want to use. At BSC, each model version appears generally under a git project: for example: https:// | ||
NB2: to launch an experiment from BSC, do not do it from your local machine, but from the machine bscesautosubmit01 (to avoid overloading your poor own machine). => type ssh -XY bscesautosubmit01 to open the connection to this machine. | NB2: to launch an experiment from BSC, do not do it from your local machine, but from the machine bscesautosubmit01 (to avoid overloading your poor own machine). => type ssh -XY bscesautosubmit01 to open the connection to this machine. | ||
+ | |||
+ | NB3: MareNosturm users, auto-ecearth3 assumes that you have a SSH alias to connect with a different user depending on the project to account the consumed hours to. Add to your .ssh/config following lines in case your project is bsc32 and your user bsc32704: | ||
+ | Host mn-bsc32 | ||
+ | HostName mn1.bsc.es | ||
+ | User bsc32704 | ||
+ | IdentityFile ~/ | ||
+ | Make sure you add all MN login nodes to the config file. See also here: [[https:// | ||
+ | |||
+ | NB4: When creating an experiment and you receive the error that some template/ | ||
**Second step: configuration** | **Second step: configuration** | ||
Line 27: | Line 46: | ||
* **conf/:** contains configurations files that you will need to fill before running your experiment: | * **conf/:** contains configurations files that you will need to fill before running your experiment: | ||
+ | * expdef_${exp}.conf: | ||
* autosubmit_${exp}.conf: | * autosubmit_${exp}.conf: | ||
- | * jobs_${exp}.conf: | + | * jobs_${exp}.conf: |
* platforms_${exp}.conf: | * platforms_${exp}.conf: | ||
* proj_${exp}.conf : resolution, number of processors for each component of the model, initial conditions, options of the different components (atmospheric, | * proj_${exp}.conf : resolution, number of processors for each component of the model, initial conditions, options of the different components (atmospheric, | ||
- | * **proj:** contains the directory model/ with all the source files of your climate model. This directory also contains the templates files. There is a version of templates for each HPC and each model version, since these files are used to provide | + | * **proj:** contains the directory model/ with all the source files of your climate model. This directory also contains the templates files. There is a version of templates for each HPC and each model version, since these files are used to transmit |
/ | / | ||
+ | |||
+ | **Test Suite** | ||
+ | |||
+ | For a first attempt it might be worth considering to copy the .conf files from a test experiment with bsc_trunk [[https:// | ||
** Small configuration issues - May 2016 ** | ** Small configuration issues - May 2016 ** | ||
Line 39: | Line 63: | ||
* BSC network is too slow for the current autosubmit current version. To avoid untimely failures, you have to specify in your .bashrc: \\ | * BSC network is too slow for the current autosubmit current version. To avoid untimely failures, you have to specify in your .bashrc: \\ | ||
export SAGA_PTY_SSH_TIMEOUT=90 \\ | export SAGA_PTY_SSH_TIMEOUT=90 \\ | ||
- | * When running on ecmwf or marenostrum, | + | * When running on ecmwf or marenostrum |
**Third step post-processing** | **Third step post-processing** | ||
Line 66: | Line 90: | ||
If the simulation is correctly finished, you will find at the end of this output file "End of Heap Utilization Profile" | If the simulation is correctly finished, you will find at the end of this output file "End of Heap Utilization Profile" | ||
+ | **How to extend an experiment** | ||
+ | |||
+ | This session explains how to run extra chunks to an experiment already finished | ||
+ | - Modify the expdef_${exp}.conf and add the chuncks you want (the total n of chuncks you want have at end) | ||
+ | - Create the experiment (autosubmit create exp_ID, see autosubmit manual for more details: [[http:// | ||
+ | - Run REMOTESETUP and INI. In order to run only these jobs set LOCALSETUP to COMPLETED and SIM1 to SUSPENDED. (use // | ||
+ | - Recovery all the jobs already completed | ||
+ | - Check that LOCALPOST and PLOT are set to WAITING, they might be COMPLETED, if the experiment was finished | ||
+ | - Transfer the restart file of the last chunk from **/ | ||
+ | The file to be transferred are: | ||
+ | * RESTC_xxxxxxx for the coupler | ||
+ | * RESTO_xxxxxxxx | ||
+ | * RESTA_xxxxxxx for the atmosphere | ||
+ | Untar these files and run the experiment as explained in the previous session | ||