From 4ead1bdb741c9b0b066e256faf59e87b3e446fe0 Mon Sep 17 00:00:00 2001 From: nperez Date: Fri, 14 Feb 2020 15:36:01 +0100 Subject: [PATCH 1/4] FAQ about path definition --- inst/doc/faq.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/inst/doc/faq.md b/inst/doc/faq.md index 0b3bc53..3fef595 100644 --- a/inst/doc/faq.md +++ b/inst/doc/faq.md @@ -11,6 +11,7 @@ This document intends to be the first reference for any doubts that you may have 5. [Do interpolation in Start() (using parameter 'transform')](#5-do-interpolation-in-start-using-parameter-transform) 6. [Get data attributes without retrieving data to workstation](#6-get-data-attributes-without-retrieving-data-to-workstation) 7. [Avoid or specify a node from cluster in Compute()](#7-avoid-or-specify-a-node-from-cluster-in-compute) + 8. [Define a path with multiple dependencies](#8-define-a-path-with-multiple-dependencies) 2. **Something goes wrong...** @@ -333,6 +334,54 @@ Look at the position of `extra_queue_params` parameter in a full call of Compute wait = TRUE) ``` +###8. Define a path with multiple dependencies + +The structure of the BSC Earth data repository, esarchive, allow us to create a path pattern to the data by using diferent variables (between dolar symbol), such as, $var$, for the variable name, or $sdates$, for the start date of the simulation. Here a example for loading montly simulations of system4_m1 data: + +path <- '/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc' + +The function Start() will require two parameters 'var' and 'sdate' to load the desired data. + +In other cases, the creation of the path could be a litle bit more complicated. Some researcher create their own EC-Earth experiments which are identified by a experiment ID ($expid$) and the are run with different model version ($version) even for different members ($member$): + +| expid | member | version | +|-------|----------|---------| +| a1st | r7i1p1f1 |v20190302| +| a1sx |r10i1p1f1 |v20190308| + +In this case, the variables member and version take different value depending on the expid (the member r10i1p1f1 does not exist for expid a1st). The path will include this varibles: + +path <- '/esarchive/exp/ecearth/$expid$/diags/CMIP/EC-Earth-Consortium/EC-Earth3/historical/$member$/Omon/$var$/gn/$version$/$var$_Omon_EC-Earth3_historical_$member$_gn_$year$.nc' + +However, the following paramters are mandatory to make Start() aware that they are not independent variables: + +``` + member_depends = 'expid', + version_depends = 'expid', +``` + +The final Start() call will look like: + +``` +yrh1 = 1960 +yrh2 = 2014 +years <- paste0(c(yrh1 : yrh2), '01-', c(yrh1 : yrh2), '12') +data <- Start(dat = repos, + var = 'tosmean', + expid = c('a1st','a1sx'), + member = 'all', + version = 'all', + member_depends = 'expid', + version_depends = 'expid', + year = years, + time = 'all', + region = indices(1 : 4), + return_vars = list(time = NULL, region = NULL), + retrieve = TRUE) +``` + + + ## Something goes wrong... ### 1. No space left on device -- GitLab From c4465bb17a2d244d9b612317a3d492b26b42e32f Mon Sep 17 00:00:00 2001 From: nperez Date: Fri, 14 Feb 2020 15:37:42 +0100 Subject: [PATCH 2/4] Missing space to FAQ title --- inst/doc/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inst/doc/faq.md b/inst/doc/faq.md index 3fef595..66b6a45 100644 --- a/inst/doc/faq.md +++ b/inst/doc/faq.md @@ -334,7 +334,7 @@ Look at the position of `extra_queue_params` parameter in a full call of Compute wait = TRUE) ``` -###8. Define a path with multiple dependencies +### 8. Define a path with multiple dependencies The structure of the BSC Earth data repository, esarchive, allow us to create a path pattern to the data by using diferent variables (between dolar symbol), such as, $var$, for the variable name, or $sdates$, for the start date of the simulation. Here a example for loading montly simulations of system4_m1 data: -- GitLab From 1a624560d3f448a4cdc8b78e3f6a4f94a404b5ec Mon Sep 17 00:00:00 2001 From: nperez Date: Fri, 14 Feb 2020 15:40:14 +0100 Subject: [PATCH 3/4] formatting FAQ about path definition --- inst/doc/faq.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/inst/doc/faq.md b/inst/doc/faq.md index 66b6a45..5097bac 100644 --- a/inst/doc/faq.md +++ b/inst/doc/faq.md @@ -336,13 +336,13 @@ Look at the position of `extra_queue_params` parameter in a full call of Compute ### 8. Define a path with multiple dependencies -The structure of the BSC Earth data repository, esarchive, allow us to create a path pattern to the data by using diferent variables (between dolar symbol), such as, $var$, for the variable name, or $sdates$, for the start date of the simulation. Here a example for loading montly simulations of system4_m1 data: +The structure of the BSC Earth data repository, esarchive, allow us to create a path pattern to the data by using diferent variables (between dolar symbol), such as, `$var$`, for the variable name, or `$sdates$`, for the start date of the simulation. Here a example for loading montly simulations of system4_m1 data: -path <- '/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc' +`path <- '/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc'` The function Start() will require two parameters 'var' and 'sdate' to load the desired data. -In other cases, the creation of the path could be a litle bit more complicated. Some researcher create their own EC-Earth experiments which are identified by a experiment ID ($expid$) and the are run with different model version ($version) even for different members ($member$): +In other cases, the creation of the path could be a litle bit more complicated. Some researcher create their own EC-Earth experiments which are identified by a experiment ID (`$expid$`) and the are run with different model version (`$version`) even for different members (`$member$`): | expid | member | version | |-------|----------|---------| @@ -351,7 +351,7 @@ In other cases, the creation of the path could be a litle bit more complicated. In this case, the variables member and version take different value depending on the expid (the member r10i1p1f1 does not exist for expid a1st). The path will include this varibles: -path <- '/esarchive/exp/ecearth/$expid$/diags/CMIP/EC-Earth-Consortium/EC-Earth3/historical/$member$/Omon/$var$/gn/$version$/$var$_Omon_EC-Earth3_historical_$member$_gn_$year$.nc' +`path <- '/esarchive/exp/ecearth/$expid$/diags/CMIP/EC-Earth-Consortium/EC-Earth3/historical/$member$/Omon/$var$/gn/$version$/$var$_Omon_EC-Earth3_historical_$member$_gn_$year$.nc'` However, the following paramters are mandatory to make Start() aware that they are not independent variables: -- GitLab From 392f5b8d1c6ade6842a4aa2fe1e932643fd103d0 Mon Sep 17 00:00:00 2001 From: aho Date: Fri, 14 Feb 2020 17:40:19 +0100 Subject: [PATCH 4/4] Correct typos. --- inst/doc/faq.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/inst/doc/faq.md b/inst/doc/faq.md index 5097bac..1ce2616 100644 --- a/inst/doc/faq.md +++ b/inst/doc/faq.md @@ -336,24 +336,24 @@ Look at the position of `extra_queue_params` parameter in a full call of Compute ### 8. Define a path with multiple dependencies -The structure of the BSC Earth data repository, esarchive, allow us to create a path pattern to the data by using diferent variables (between dolar symbol), such as, `$var$`, for the variable name, or `$sdates$`, for the start date of the simulation. Here a example for loading montly simulations of system4_m1 data: +The structure of the BSC Earth data repository 'esarchive' allows us to create a path pattern to the data by using different variables (between dollar symbol), such as `$var$`, for the variable name, or `$sdates$`, for the start date of the simulation. Here is an example for loading monthly simulations of system4_m1 data: `path <- '/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc'` The function Start() will require two parameters 'var' and 'sdate' to load the desired data. -In other cases, the creation of the path could be a litle bit more complicated. Some researcher create their own EC-Earth experiments which are identified by a experiment ID (`$expid$`) and the are run with different model version (`$version`) even for different members (`$member$`): +In some cases, the creation of the path could be a little bit more complicated. Some researchers create their own EC-Earth experiments which are identified by an experiment ID (`$expid$`) and with different model version (`$version`), even for different members (`$member$`): | expid | member | version | |-------|----------|---------| | a1st | r7i1p1f1 |v20190302| | a1sx |r10i1p1f1 |v20190308| -In this case, the variables member and version take different value depending on the expid (the member r10i1p1f1 does not exist for expid a1st). The path will include this varibles: +In this case, the variable member and version have different value depending on the expid (the member r10i1p1f1 does not exist for expid a1st). The path will include this varibles: `path <- '/esarchive/exp/ecearth/$expid$/diags/CMIP/EC-Earth-Consortium/EC-Earth3/historical/$member$/Omon/$var$/gn/$version$/$var$_Omon_EC-Earth3_historical_$member$_gn_$year$.nc'` -However, the following paramters are mandatory to make Start() aware that they are not independent variables: +However, the following parameters are mandatory to make Start() aware of that they are not independent variables: ``` member_depends = 'expid', -- GitLab