|
|
# Context
|
|
|
|
|
|
Not quite infrequently, experiments must be transferred from their original workspace (scratch storage and any other partitions used by the workflow) to any other place before they finish their execution.
|
|
|
|
|
|
Some situations in which this could happen are:
|
|
|
|
|
|
* The computing resources (RES, PRACE, etc.) allocation comes to an end.
|
|
|
* The experiment's owner needs to transfer their experiment to another user.
|
|
|
|
|
|
Manually transferring experiments is prone to errors, can leave the experiment in a deadlock situation (without the option to resume it), or lead to irrecoverable data losses. Besides, it can be an energy and time-demanding task.
|
|
|
|
|
|
Autosubmit has had this `migrate` feature from its early years. However, the feature had some limitations so, in 2021, this tool has been largely improved and tested, adding mechanisms to increase the efficiency and robustness of the move to adequate it to the current user's needs.
|
|
|
|
|
|
Below these lines, there are comprehensive instructions about performing this delicate operation and information about the tests used to verify the correct operation.
|
|
|
|
|
|
# Before the migration
|
|
|
|
|
|
Before starting the migation, the following actions are recommended:
|
... | ... | @@ -26,7 +39,7 @@ The refactored tool was successfully tested with three **Auto-EC-Earth** and one |
|
|
| a3kj | Auto-EC-Earth 3.3.3_production (1) | 1 startdate, 3 yearly chunks | bsc32 | RES | Same bsc32 user |
|
|
|
| t0jd | Auto-MONARCH master | 2 startdates, 4 members, 10 daily chunks | RES | PRACE | Same bsc32 user |
|
|
|
|
|
|
(1) _3.3.3_production_ was patched the 11/03/2021 with a [fix](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/commit/0d66c71f85671786ad6032c13b2dcda527a83254) that removed the requirement of changing the ownership of _/esarchive/exp/ecearth/${EXPID}_. Experiments running _3.3.3_production_ need to get that patch (only affecting the timestamp of the directories) to avoid problems if the experiment already ran the first three chunks.
|
|
|
(1) _3.3.3_production_ was patched the 11/03/2021 with a [fix](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/commit/0d66c71f85671786ad6032c13b2dcda527a83254) that removed the requirement of changing the ownership of _/esarchive/exp/ecearth/${EXPID}_. Therefore, experiments running _3.3.3_production_ need to get that patch (only affecting the timestamp of the directories) to avoid problems if the experiment already ran the first three chunks.
|
|
|
|
|
|
_a3ke_ (copy of a3k1) was entirely run with a bsc32 user, and the output successfully checked with _a3k1_ using the output checker.
|
|
|
|
... | ... | @@ -44,7 +57,7 @@ After performing the [recommended actions](https://earth.bsc.es/gitlab/es/autosu |
|
|
|
|
|
* *a3h5* was updated with 3.3.3_production head, to updated it with the aforementioned [patch](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/commit/0d66c71f85671786ad6032c13b2dcda527a83254).
|
|
|
* *a3h7* was also received a similar [patch](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/issues/1526#note_127011).
|
|
|
* *a3fy* was the experiment running the oldest version. It had to be updated in a similar fashion with this ad-hoc [change](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/issues/1526#note_127004). It also needed a further [patch](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/blob/d0f973acdb5b26a5fee681c996bfd40da3d1b649/plugins/utils.sh#L13) on `TRANSFER` to avoid changing parents directories recursively.
|
|
|
* *a3fy* was the experiment running the oldest version. It had to be updated similarly with this ad-hoc [change](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/issues/1526#note_127004). It also needed a further [patch](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/blob/d0f973acdb5b26a5fee681c996bfd40da3d1b649/plugins/utils.sh#L13) on `TRANSFER` to avoid changing parents directories recursively.
|
|
|
|
|
|
|
|
|
|
|
|
\ No newline at end of file |