User Tools

Site Tools


scalability_with_autosubmit

This is an old revision of the document!


Automate a scalability analysis with Autosubmit

In case of having to perform a bi-dimensional scalability analysis, a lot of simulations are needed. In order to deal with this, it is possible to automate the whole analysis with the ec-earth_autosubmit_perf.sh script. It is necessary to modify some variables to adapt the scalability analysis, such as the number of MPI tasks (in the two bucles), the MareNostrum's username, all the initialization variables, etc.

On the other hand, once experiments have finsihed, it is possible to gather the execution times using ec-earth_autosubmit_perf_results.sh script. The previous script will generate a file called “experiments-ids.list” which contains all the experiments identifiers. It is needed to process all the execution times to output the results. Furthermore, it is necessary to set properly the “RESULTS_BASE_DIR” variable. Once executed, the script generates two files called “execution_times.txt” and “usage_resources_in_time.txt” containing execution times and total CPU hours used respectively for each combination of MPI tasks.

Finally, since we are only interested in execution times, it is possible to remove outputs to save disk space. The clean_outputs.sh script removes the run directory, but keeps the logs. It also uses the “experiments-ids.list” and it is necessary to set the “RESULTS_BASE_DIR” variable.

Note that all the scripts are coded to perform the average of 3 executions for each combination of MPI tasks.

All this was tested with version 3.2.0 of Autosubmit. With newer versions it could not work. For any problem or question, please contact me: xavier.yepes@bsc.es

scalability_with_autosubmit.1481789830.txt.gz · Last modified: 2016/12/15 08:17 by xyepes