Autosubmit is a tool to create, manage and monitor experiments by using different (configured) HPC's/Clusters remotely via ssh. HOW TO DEPLOY/SETUP AUTOSUBMIT FRAMEWORK ======================================== - Autosubmit has been tested: with the following Operating Systems: * Linux Debian on the following HPC's/Clusters: * Ithaca (IC3's Cluster) * MareNostrum (Barcelona Supercomputing Center) * HECToR (UK Based supercomputer) * Lindgren (Swedish machine) - Pre-requisties: These packages (python2, python-argparse, python-dateutil, python-pydot, python-matplotlib, sqlite3) must be available at local machine. And the machine is also able to access HPC's/Clusters via password-less ssh. - Create a repository for experiments: Say for expample "/cfu/autosubmit" then edit the repository path into src/dir_config.py, src/expid.py, conf/autosubmit.conf - Create a blank database: Say for example "autosubmit.db" at above created repository and thereafter: > cd /cfu/autosubmit > sqlite3 autosubmit.db sqlite3>.read ../../src/autosubmit.sql > chmod 777 autosubmit.db HOW TO USE AUTOSUBMIT ===================== > cd src > python expid.py -h > python expid.py --new ecearth --HPC ithaca --description "experiment is about..." Say for example, "chex" is 4 character based expid generated by system. First character "c" represents the platform such as "i" for ithaca, "b" for bsc, "h" for hector etc. While the rest of three characters are to represent unique hexadecimal number for the experiment. > vi /cfu/autosubmit/chex/conf/expdef_chex.conf > vi /cfu/autosubmit/chex/conf/autosubmit_chex.conf > python create_exp.py chex > ./setupexp.sh -e chex > nohup python autosubmit.py chex >& chex_01.log & HOW TO MONITOR EXPERIMENT ========================= > cd src > python monitor.py -h > python monitor.py -e chex -j job_list -o pdf or > python monitor.py -e chex -j job_list -o png Above generated plot with date & time stamp can be found at: /cfu/autosubmit/chex/plot/chex_date_time.pdf or /cfu/autosubmit/chex/plot/chex_date_time.png HOW TO RESTART EXPERIMENT ========================= > cd src > python recovery.py -h > python recovery.py -e chex -j job_list -g # getting/fetching completed files > python recovery.py -e chex -j job_list -s # saving the pickle file > nohup python autosubmit.py chex >& chex_02.log & HOW TO RERUN/EXTEND EXPERIMENT ============================== > cd src > vi /cfu/autosubmit/chex/conf/expdef_chex.conf # modify RERUN, CHUNKLIST > python create_exp.py chex > nohup python autosubmit.py chex >& chex_03.log & Monitor for RERUN ------------------ > python monitor.py -e chex -j rerun_job_list -o pdf Recovery for RERUN ------------------- > python recovery.py -e chex -j rerun_job_list -g > python recovery.py -e chex -j rerun_job_list -s