Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • autosubmit autosubmit
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 338
    • Issues 338
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 21
    • Merge requests 21
  • Deployments
    • Deployments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Earth SciencesEarth Sciences
  • autosubmitautosubmit
  • Wiki
  • migrate experiments

migrate experiments · Changes

Page history
Update migrate experiments authored Jun 03, 2021 by Miguel Castrillo's avatar Miguel Castrillo
Hide whitespace changes
Inline Side-by-side
migrate-experiments.md
View page @ ddefd78f
...@@ -22,16 +22,16 @@ Before starting the migation, the following actions are recommended: ...@@ -22,16 +22,16 @@ Before starting the migation, the following actions are recommended:
# Configure your experiment for the migration # Configure your experiment for the migration
In order to migrate an experiment, an configuration of the migrate parameters for EACH platform that the experiment uses is necessary. You must set all migrate parameters on platforms_expid.conf. In order to migrate an experiment, it is necessary to configure the migrate options for EACH platform used by the experiment workflow. Therefore, you must set all migrate parameters on platforms_expid.conf.
The mandatory configuration consists of: The mandatory configuration consists of:
* USER_TO = <target_user> * USER_TO = <target_user>
* TEMP_DIR = <hpc_temporary_directory> * TEMP_DIR = <hpc_temporary_directory>
Temp_dir is a dir on the remote platform to which both users must have access. Ideally, this dir should have these permissions (RWX|RWX|---), however (RWX|RWX|RWX) also works. `TEMP_DIR` is a dir on the remote platform to which both users must have access. Ideally, this dir should have these permissions: `rwx|rwx|---`; however, `rwx|rwx|rwx` also works.
Another important point to have into account is that if your experiment data is in a shared filesystem ( example, dt-transfer and marenostrum4) you only have to fill **TEMP_DIR** on the dt-transfer platform. In case your experiment data is in a filesystem shared for different platforms (for example, DT nodes and MareNostrum4) you only have to fill **TEMP_DIR** on one of the transfer platforms (DT in this case).
ex: ex:
``` ```
...@@ -60,26 +60,26 @@ USER_TO = bsc32523 ...@@ -60,26 +60,26 @@ USER_TO = bsc32523
TEMP_DIR = /gpfs/scratch/bsc32/bsc32070/temp_dir TEMP_DIR = /gpfs/scratch/bsc32/bsc32070/temp_dir
``` ```
Finally, is mandatory that this folder is in the **same filesystem**. Finally, this folder must be in the **same filesystem**.
You can expand this **basic configuration** as follow: You can expand this **basic configuration** as follows:
* SAME_USER = false|true # Default, FALSE, used for mantain the same remote_user * SAME_USER = false|true # Default, false, used only when the remote user is the same
* PROJECT_TO = <project> # Optional, if not specified project will remain the same * PROJECT_TO = <project> # Optional, if not specified, project (group) will remain the same
* HOST_TO = <cluster_ip> # Optional, avoid alias if possible, try use direct ip. * HOST_TO = <cluster_ip> # Optional, avoid alias if possible, try use direct ip.
**Before running** the experiment with the new user, remind them to change the **queue** parameter (example CLASS_A to bsc_es) if necessary. **Before running** the experiment with the new user, remind to change the **queue** parameter (example CLASS_A to bsc_es) if necessary.
# Perform the migration # Perform the migration
Once configured, double-check that you're using the AS v3.13.0+. Once configured, double-check that you're using the AS v3.13.0+ or a more recent one.
There are two modes of running `autosubmit migrate`, both of them consist of a two-step procedure: There are two modes of running `autosubmit migrate`, both of them consist of a two-step procedure:
* Migrate everything: * Migrate everything:
* * (User-Owner) `autosubmit migrate <expid> -o` * * (User-Owner) `autosubmit migrate <expid> -o`
* * (User-Target)`autosubmit migrate <expid> -p` * * (User-Target)`autosubmit migrate <expid> -p`
* Migrate only remote files. (usefull if the local user is the same) * Migrate only remote files (usefull if the local user is the same).
* * (User-Owner) `autosubmit migrate <expid> -o --onlyremote` * * (User-Owner) `autosubmit migrate <expid> -o --onlyremote`
* * (User-Target) `autosubmit migrate <expid> -p --onlyremote` * * (User-Target) `autosubmit migrate <expid> -p --onlyremote`
......
Clone repository
  • Code coverage
  • Deployment
  • Issues documenting different aspects
  • Leaflet
  • Possible Operational Problems and Solutions
  • Running Autosubmit in Earth Sciences
  • Testing_Suite
  • Updating ReadTheDocs Autosubmit documentation
  • Visual Identity
  • [DestinE] Autosubmit VM on Lumi
  • background
  • bibtex
  • databases
  • development
  • dissemination
View All Pages