diff --git a/inst/doc/tutorial/PATC2021/Figures/.gitkeep b/inst/doc/tutorial/PATC2021/Figures/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/inst/doc/tutorial/PATC2021/Figures/handson_1_fig_crps.png b/inst/doc/tutorial/PATC2021/Figures/handson_1_fig_crps.png new file mode 100644 index 0000000000000000000000000000000000000000..73c480a848755ce57ac162fbc7d71e51e89c8329 Binary files /dev/null and b/inst/doc/tutorial/PATC2021/Figures/handson_1_fig_crps.png differ diff --git a/inst/doc/tutorial/PATC2021/Figures/handson_1_fig_rmsss.png b/inst/doc/tutorial/PATC2021/Figures/handson_1_fig_rmsss.png new file mode 100644 index 0000000000000000000000000000000000000000..7086350961e47067a018bdef9ad4fd4abdca5703 Binary files /dev/null and b/inst/doc/tutorial/PATC2021/Figures/handson_1_fig_rmsss.png differ diff --git a/inst/doc/tutorial/PATC2021/PATC2021_handson.pdf b/inst/doc/tutorial/PATC2021/PATC2021_handson.pdf new file mode 100644 index 0000000000000000000000000000000000000000..5630f10ab6daa4a3ecbc02a02d62482b579854c5 Binary files /dev/null and b/inst/doc/tutorial/PATC2021/PATC2021_handson.pdf differ diff --git a/inst/doc/tutorial/PATC2021/hands-on_1_workflow.md b/inst/doc/tutorial/PATC2021/hands-on_1_workflow.md new file mode 100644 index 0000000000000000000000000000000000000000..4f91a51f3a4060bf5b676c5e1ee76fd6301fd9d5 --- /dev/null +++ b/inst/doc/tutorial/PATC2021/hands-on_1_workflow.md @@ -0,0 +1,160 @@ +# Hands-on 1: Seasonal forecast verification + +## Goal +Learn how to use the startR workflow to finish a piece of analysis, including defining and pre-processing the desired data, +defining the operation, building the workflow, and executing the operation. + +In this use case, the ensemble-adjusted Continuous Ranked Probability Score (CRPS) and the root mean square error skill score (RMSSS) are computed to verify the forecast. +To make the process faster, the required data size is small here so we can run the execution on workstation. + +## 1. Define data +The first step of the analysis is use Start() to identify the data needed. + +Hints: +(1) The data paths use two '$' as the wildcard, and it can be given any names you like. +(2) The variable we want to use is 'tas'. +(3) The required time (sdate) period is November 1981 to 2016. +(4) Read global data (i.e., full spatial grids.) +(5) Take only the first time step. +(6) To make the data smaller, we only take the first two ensemble members from the experimental data. +(7) Because we are going to use the whole startR workflow, we only need to create a pointer to the data repository rather than retrieve it to the workstation. + +```r +library(startR) + +# exp data + # Use this one if on workstation or nord3 + repos_exp <- paste0('/esarchive/exp/ecmwf/system4_m1/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + # Use this one if on Marenostrum4 and log in with PATC2021 account + repos_exp <- paste0('/gpfs/scratch/nct00/nct00016/d3_02_R-hands_on/tmp-esarchive/', + 'exp/ecmwf/system4_m1/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + + sdates_exp <- sapply(1981:2016, function(x) paste0(x, '1101')) + + exp <- Start(dat = repos_exp, + var = ______, + sdate = sdates_exp, + time = ______, + ensemble = ______, + latitude = ______, + longitude = ______, + synonims = list(longitude = c('lon', 'longitude'), + latitude = c('lat', 'latitude')), + return_vars = list(latitude = NULL, longitude = NULL, time = NULL), + retrieve = ______) + +# obs data + repos_obs <- paste0('/gpfs/scratch/nct00/nct00016/d3_02_R-hands_on/tmp-esarchive/', + 'recon/ecmwf/erainterim/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + sdates_obs <- sapply(1981:2016, function(x) paste0(x, '11')) + + obs <- Start(dat = repos_obs, + var = ______, + sdate = sdates_obs, + time = ______, + latitude = ______, + longitude = ______, + synonims = list(longitude = c('lon', 'longitude'), + latitude = c('lat', 'latitude')), + return_vars = list(latitude = NULL, longitude = NULL, time = NULL), + retrieve = ______) +``` + +Question: + +1. What are the dimensions of these two data? + +2. What is the size of these two data? + + +## 2. Define operation and workflow +It is recommended to define the function and write Step() together, because the latter one helps you clarify the input and output dimensions of the function. + +In the self-defined function, we want to use the two functions, 'SpecsVerification::EnsCrps' and 's2dv:::.RMSSS', to calculate the skill score. +You can check the first function by typing `?SpecsVerification::EnsCrps`, and the second function on [s2dv GitLab](https://earth.bsc.es/gitlab/es/s2dv/-/blob/master/R/RMSSS.R). + +Hint: +(1) The self-defined function is for exp and obs data. Therefore, target_dims and output_dims in Step() should be a list with two vector elements. +(2) Check the functions 'EnsCrps' and '.RMSSS'. What are the minimum required dimensions of inputs of each function? These dimensions should be the 'target_dims' in Step(). +(3) To return the two skill scores ('crps' and 'rmsss') together, put them in a list with two elements. +(4) What are the dimensions of outputs? These dimensions should be the 'output_dims' in Step(). +(5) The first input of AddStep() should also be a list containing exp and obs. + +```r + # self-defined function + func <- function(exp, obs) { + # CRPS + crps <- mean(SpecsVerification::EnsCrps(exp, obs, R.new = Inf), na.rm = TRUE) + + # RMSSS + # obs only has [sdate] now. Add one more dim for it. + obs <- s2dv::InsertDim(obs, posdim = 2, lendim = 1, name = 'ensemble') + rmsss <- mean(s2dv:::.RMSSS(exp, obs, time_dim = 'sdate', dat_dim = 'ensemble', pval = FALSE)$rmsss) + + return(______) + } + step <- Step(func, target_dims = ______, + output_dims = ______) + wf <- AddStep(______, step) +``` + +Question: +1. Which dimensions are used in operation? Which dimensions are free? + +2. Can we use more dimensions as target_dims? What are the disadvantages? + + +## 3. Execute locally +To avoid potential technical problems in the connection and configuration, +we choose to run the execution locally. +Noted that it is recommended to submit jobs to HPCs if the data size is big or +the operation is heavy. + +Hint: +(1) Use the free dimensions (i.e., those are not 'target_dims' in Step()) to chunk. +(2) It is safe to divide the data into pieces of which the size each is 1/2-1/3 of RAM. In this case, the data size is only around 100Mb, so you can play with it without +worrying about crashing the workstation. +(3) You will see some error messages like "Error in R_nc4_open: No such file or directory", +which are ignorable. + +```r + res <- Compute(wf$crps, + chunks = list(______)) +``` + +## 4. Check the results + +1. Check the list structure using function `str()`. What are the items in the list? + +2. What do the dimensions of each item look like? + +3. Plot the map +Use `s2dv::PlotEquiMap` or other visualization tools to plot the map. +```r + # Get latitude and longitude from the attributes + lat <- as.vector(attr(exp, 'Variables')$common$latitude) + lon <- as.vector(attr(exp, 'Variables')$common$longitude) + + # Set the color bar + brks_crps <- seq(min(res$crps, na.rm = TRUE), max(res$crps, na.rm = TRUE), length.out = 11) + brks_rmsss <- seq(min(res$rmsss, na.rm = TRUE), max(res$rmsss, na.rm = TRUE), length.out = 11) + + library(s2dv) + # Plot crps + PlotEquiMap(______, lon, lat, + color_fun = clim.palette('yellowred'), brks = brks_crps, + filled.continents = FALSE, triangle_ends = c(TRUE, TRUE), + toptitle = 'ECMWF monthly mean tas CRPS 2012-2016', title_scale = 0.6, + fileout = '~/handson_1_fig_crps.png') + + # Plot rmsss + PlotEquiMap(______, lon, lat, + color_fun = clim.palette('yellowred'), brks = brks_rmsss, + filled.continents = FALSE, triangle_ends = c(TRUE, TRUE), + toptitle = 'ECMWF monthly mean tas RMSSS 2012-2016', title_scale = 0.6, + fileout = '~/handson_1_fig_rmsss.png') + +``` diff --git a/inst/doc/tutorial/PATC2021/hands-on_1_workflow_ans.md b/inst/doc/tutorial/PATC2021/hands-on_1_workflow_ans.md new file mode 100644 index 0000000000000000000000000000000000000000..5ecc054d115afd533c6b58d669895ffee041dc03 --- /dev/null +++ b/inst/doc/tutorial/PATC2021/hands-on_1_workflow_ans.md @@ -0,0 +1,193 @@ +# Hands-on 1: Seasonal forecast verification + +## Goal +Learn how to use the startR workflow to finish a piece of analysis, including defining and pre-processing the desired data, +defining the operation, building the workflow, and executing the operation. + +In this use case, the ensemble-adjusted Continuous Ranked Probability Score (CRPS) and the root mean square error skill score (RMSSS) are computed to verify the forecast. +To make the process faster, the required data size is small here so we can run the execution on workstation. + +## 1. Define data +The first step of the analysis is use Start() to identify the data needed. + +Hints: +(1) The data paths use two '$' as the wildcard, and it can be given any names you like. +(2) The variable we want to use is 'tas'. +(3) The required time (sdate) period is November 1981 to 2016. +(4) Read global data (i.e., full spatial grids.) +(5) Take only the first time step. +(6) To make the data smaller, we only take the first two ensemble members from the experimental data. +(7) Because we are going to use the whole startR workflow, we only need to create a pointer to the data repository rather than retrieve it to the workstation. + +```r +library(startR) + +# exp data + # Use this one if on workstation or nord3 + repos_exp <- paste0('/esarchive/exp/ecmwf/system4_m1/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + # Use this one if on Marenostrum4 and log in with PATC2021 account + repos_exp <- paste0('/gpfs/scratch/nct00/nct00016/d3_02_R-hands_on/tmp-esarchive/', + 'exp/ecmwf/system4_m1/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + + sdates_exp <- sapply(1981:2016, function(x) paste0(x, '1101')) + + exp <- Start(dat = repos_exp, + var = 'tas', + sdate = sdates_exp, + time = indices(1), + ensemble = indices(1:2), + latitude = 'all', + longitude = 'all', + synonims = list(longitude = c('lon', 'longitude'), + latitude = c('lat', 'latitude')), + return_vars = list(latitude = NULL, longitude = NULL, time = 'sdate'), + retrieve = FALSE) + +# obs data + repos_obs <- paste0('/gpfs/scratch/nct00/nct00016/d3_02_R-hands_on/tmp-esarchive/', + 'recon/ecmwf/erainterim/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + sdates_obs <- sapply(1981:2016, function(x) paste0(x, '11')) + + obs <- Start(dat = repos_obs, + var = 'tas', + sdate = sdates_obs, + time = indices(1), + latitude = 'all', + longitude = 'all', + synonims = list(longitude = c('lon', 'longitude'), + latitude = c('lat', 'latitude')), + return_vars = list(latitude = NULL, longitude = NULL, time = 'sdate'), + retrieve = FALSE) +``` + +Question: + +1. What are the dimensions of these two data? +```r +attr(exp, 'Dimensions') + dat var sdate time ensemble latitude longitude + 1 1 36 1 2 256 512 + +attr(obs, 'Dimensions') + dat var sdate time latitude longitude + 1 1 36 1 256 512 +``` + +2. What is the size of these two data? +exp: 72Mb; obs: 36Mb + + +## 2. Define operation and workflow +It is recommended to define the function and write Step() together, because the latter one helps you clarify the input and output dimensions of the function. + +In the self-defined function, we want to use the two functions, 'SpecsVerification::EnsCrps' and 's2dv:::.RMSSS', to calculate the skill score. +You can check the first function by typing `?SpecsVerification::EnsCrps`, and the second function on [s2dv GitLab](https://earth.bsc.es/gitlab/es/s2dv/-/blob/master/R/RMSSS.R). + +Hint: +(1) The self-defined function is for exp and obs data. Therefore, target_dims and output_dims in Step() should be a list with two vector elements. +(2) Check the functions 'EnsCrps' and '.RMSSS'. What are the minimum required dimensions of inputs of each function? These dimensions should be the 'target_dims' in Step(). +(3) To return the two skill scores ('crps' and 'rmsss') together, put them in a list with two elements. +(4) What are the dimensions of outputs? These dimension should be the 'output_dims' in Step(). +(5) The first input of AddStep() should also be a list containing exp and obs. + +```r + # self-defined function + func <- function(exp, obs) { + # CRPS + crps <- mean(SpecsVerification::EnsCrps(exp, obs, R.new = Inf), na.rm = TRUE) + + # RMSSS + # obs only has [sdate] now. Add one more dim for it. + obs <- s2dv::InsertDim(obs, posdim = 2, lendim = 1, name = 'ensemble') + rmsss <- mean(s2dv:::.RMSSS(exp, obs, time_dim = 'sdate', dat_dim = 'ensemble', pval = FALSE)$rmsss) + + return(list(crps = crps, rmsss = rmsss)) + } + step <- Step(func, target_dims = list(c('sdate', 'ensemble'), c('sdate')), + output_dims = list(crps = NULL, rmsss = NULL)) + wf <- AddStep(list(exp, obs), step) +``` + +Question: +1. Which dimensions are used in operation? Which dimensions are free? +'sdate' and 'ensemble' are used in operation. The other dimensions (dat, var, time, latitude, and longitude) are not necessary for the function. These free dimensions will be subsetted by multiApply during the execution. + +2. Can we use more dimensions as target_dims? What are the disadvantages? +It still works if putting more dimensions as target_dims, but we will lose the choices for chunking in the next step. Also, the lighter the function is, the quicker the operation runs. + + + +## 3. Execute locally +To avoid potential technical problems in the connection and configuration, +we choose to run the execution locally. +Noted that it is recommended to submit jobs to HPCs if the data size is big or +the operation is heavy. + +Hint: +(1) Use the free dimensions (i.e., those are not 'target_dims' in Step()) to chunk. +(2) It is safe to divide the data into pieces of which the size each is 1/2-1/3 of RAM. In this case, the data size is only around 100Mb, so you can play with it without +worrying about crashing the workstation. +(3) You will see some error messages like "Error in R_nc4_open: No such file or directory", +which are ignorable. + +```r + res <- Compute(wf$crps, + chunks = list(latitude = 2, longitude = 2)) +``` + +## 4. Check the results + +1. Check the list structure using function `str()`. What are the items in the list? +```r +str(res) +List of 2 + $ crps : num [1, 1, 1, 1:256, 1:512] 0.879 0.97 1.029 1.04 0.991 ... + $ rmsss: num [1, 1, 1, 1:256, 1:512] 0.991 0.991 0.99 0.99 0.99 ... + ... +``` + +2. What does the dimension look like? +```r +dim(res$crps) + dat var time latitude longitude + 1 1 1 256 512 +dim(res$rmsss) + dat var time latitude longitude + 1 1 1 256 512 +``` + +3. Plot the map +Use `s2dv::PlotEquiMap` or other visualization tools to plot the map. +```r + # Get latitude and longitude from the attributes + lat <- as.vector(attr(exp, 'Variables')$common$latitude) + lon <- as.vector(attr(exp, 'Variables')$common$longitude) + + # Set the color bar + brks_crps <- seq(0, max(res$crps, na.rm = TRUE), length.out = 21) + brks_rmsss <- seq(min(res$rmsss, na.rm = TRUE), max(res$rmsss, na.rm = TRUE), length.out = 21) + + library(s2dv) + # Plot crps + PlotEquiMap(res$crps[1, 1, 1, , ], lon, lat, + cols = heat.colors(length(brks_crps) - 1)[(length(brks_crps) - 1):1], + brks = brks_crps, + filled.continents = FALSE, triangle_ends = c(TRUE, TRUE), + toptitle = 'ECMWF monthly mean tas CRPS 2012-2016', title_scale = 0.6, + fileout = '~/handson_1_fig_crps.png') + + # Plot rmsss + PlotEquiMap(res$rmsss[1, 1, 1, , ], lon, lat, + color_fun = clim.palette('yellowred'), brks = brks_rmsss, + filled.continents = FALSE, triangle_ends = c(TRUE, TRUE), + toptitle = 'ECMWF monthly mean tas RMSSS 2012-2016', title_scale = 0.6, + fileout = '~/handson_1_fig_rmsss.png') +``` + +![CRPS map](./Figures/handson_1_fig_crps.png) + +![RMSSS map](./Figures/handson_1_fig_rmsss.png) + diff --git a/inst/doc/tutorial/PATC2021/hands-on_2_interpolation.md b/inst/doc/tutorial/PATC2021/hands-on_2_interpolation.md new file mode 100644 index 0000000000000000000000000000000000000000..02ac7264363bb92085f679575e047a3ab4e0605f --- /dev/null +++ b/inst/doc/tutorial/PATC2021/hands-on_2_interpolation.md @@ -0,0 +1,90 @@ +# Hands-on 2: Spatial grid interpolation + +## Goal +To learn how to use Start() to load the data and do the interpolation. +In this use case, we only use Start() and load the data in the local memory. +The default transformation function is startR::CDORemapper, a wrapper function of s2dv::CDORemap that uses cdo inside. + +In detail, we are going to: +(1) Learn different ways to assign longitude and latitude in Start(). +(2) Use parameter '_reorder' to specify the sorted order. +(3) Use transform-related parameters. + +## 1. Define longitude and latitude +```r + # Use this one if on workstation or nord3 + repos <- "/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" + # Use this one if on Marenostrum4 and log in with PATC2021 account + repos <- paste0('/gpfs/scratch/nct00/nct00016/d3_02_R-hands_on/tmp-esarchive/', + 'exp/ecmwf/system5_m1/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + var <- 'tas' + sdate <- c('20170101') + lon.min <- 0 + lon.max <- 359.9 + lat.min <- -90 + lat.max <- 90 + + data <- Start(dat = repos, + var = var, + sdate = sdate, + ensemble = indices(1), + time = 'all', + latitude = values(list(lat.min, lat.max)), + latitude_reorder = Sort(), + longitude = values(list(lon.min, lon.max)), + longitude_reorder = CircularSort(0, 360), + synonims = list(latitude = c('lat', 'latitude'), + longitude = c('lon', 'longitude')), + return_vars = list(time = 'sdate', + longitude = 'dat', + latitude = 'dat'), + retrieve = TRUE + ) +``` + +1. Run the above script. Check the dimensions, the warning messages, and the values of +longitude and latitude. What is the range of longitude and latitude? + +2. Why 'lon.max' is 359.9 but not 360? What will happen if it is 360? + +3. Now, change + - `latitude_reorder = Sort()` to `latitude_reorder = Sort(decreasing = TRUE)` + - `longitude_reorder = CircularSort(0, 360)` to `longitude_reorder = CircularSort(-180, 180)` + - Set `lon.min <- -180` and `lon.max <- 179.9` + +Check the values of longitude and latitude again. Is it different from the original script? + + +## 2. Interpolation +Now, let us add in the transformation parameters. +```r + data <- Start(dat = repos, + var = var, + sdate = sdate, + ensemble = indices(1), + time = 'all', + latitude = values(list(lat.min, lat.max)), + latitude_reorder = Sort(), + longitude = values(list(lon.min, lon.max)), + longitude_reorder = CircularSort(0, 360), + ## transformation + transform = CDORemapper, + transform_extra_cells = 2, + transform_params = list(grid = 'r360x181', + method = 'conservative', + crop = c(lon.min, lon.max, + lat.min, lat.max)), + transform_vars = c('latitude', 'longitude'), + apply_indices_after_transform = FALSE, + ## + synonims = list(latitude = c('lat', 'latitude'), + longitude = c('lon', 'longitude')), + return_vars = list(time = 'sdate', + longitude = NULL, + latitude = NULL), + retrieve = TRUE + ) +``` +4. Run the above script. Check the dimensions and the values of longitude and latitude. + diff --git a/inst/doc/tutorial/PATC2021/hands-on_2_interpolation_ans.md b/inst/doc/tutorial/PATC2021/hands-on_2_interpolation_ans.md new file mode 100644 index 0000000000000000000000000000000000000000..90bac148db8d1c81a4425e5f6cc74beaac907b9d --- /dev/null +++ b/inst/doc/tutorial/PATC2021/hands-on_2_interpolation_ans.md @@ -0,0 +1,137 @@ +# Hands-on 2: Spatial grid interpolation + +## Goal +To learn how to use Start() to load the data and do the interpolation. +In this use case, we only use Start() and load the data in the local memory. +The default transformation function is startR::CDORemapper, a wrapper function of s2dv::CDORemap that uses cdo inside. + +In detail, we are going to: +(1) Learn different ways to assign longitude and latitude in Start(). +(2) Use parameter '_reorder' to specify the sorted order. +(3) Use transform-related parameters. + +## 1. Define longitude and latitude +```r + # Use this one if on workstation or nord3 + repos <- "/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" + # Use this one if on Marenostrum4 and log in with PATC2021 account + repos <- paste0('/gpfs/scratch/nct00/nct00016/d3_02_R-hands_on/tmp-esarchive/', + 'exp/ecmwf/system5_m1/monthly_mean/', + '$var$_f6h/$var$_$sdate$.nc') + var <- 'tas' + sdate <- c('20170101') + lon.min <- 0 + lon.max <- 359.9 + lat.min <- -90 + lat.max <- 90 + + data <- Start(dat = repos, + var = var, + sdate = sdate, + ensemble = indices(1), + time = 'all', + latitude = values(list(lat.min, lat.max)), + latitude_reorder = Sort(), + longitude = values(list(lon.min, lon.max)), + longitude_reorder = CircularSort(0, 360), + synonims = list(latitude = c('lat', 'latitude'), + longitude = c('lon', 'longitude')), + return_vars = list(time = 'sdate', + longitude = 'dat', + latitude = 'dat'), + retrieve = TRUE + ) +``` + +1. Run the above script. Check the dimensions, the warning messages, and the values of +longitude and latitude. What is the range of longitude and latitude? +```r +dim(data) + dat var sdate ensemble time latitude longitude + 1 1 1 1 7 640 1296 + +longitude <- attr(data, 'Variables')$dat1$longitude +range(longitude) +[1] 0.0000 359.7222 + +latitude <- attr(data, 'Variables')$dat1$latitude +latitude[1] +[1] -89.78488 +latitude[640] +[1] 89.78488 +``` + +2. Why 'lon.max' is 359.9 but not 360? What will happen if it is 360? +If it is 360, you only get one point of longitude. Because of `longitude_reorder = CircularSort(0, 360)`, Start() regards 0 and 360 as the same point. Therefore, we need to set a number slightly smaller than 360 but bigger than the maximum value in the original data, so we can get the whole range. + +3. Now, change + - `latitude_reorder = Sort()` to `latitude_reorder = Sort(decreasing = TRUE)` + - `longitude_reorder = CircularSort(0, 360)` to `longitude_reorder = CircularSort(-180, 180)` + - Set `lon.min <- -180` and `lon.max <- 179.9` + +Check the values of longitude and latitude again. Is it different from the original script? +```r +dim(data) + dat var sdate ensemble time latitude longitude + 1 1 1 1 7 640 1296 + +longitude <- attr(data, 'Variables')$dat1$longitude +range(longitude) +[1] -180.0000 179.7222 + +latitude <- attr(data, 'Variables')$dat1$latitude +latitude[1] +[1] 89.78488 +latitude[640] +[1] -89.78488 + +``` +The dimensions are the same. The longitude range changes to [-180, 180], and the latitude sorts from 90 to -90. + + +## 2. Interpolation +Now, let us add in the transformation parameters. + +```r + data <- Start(dat = repos, + var = var, + sdate = sdate, + ensemble = indices(1), + time = 'all', + latitude = values(list(lat.min, lat.max)), + latitude_reorder = Sort(), + longitude = values(list(lon.min, lon.max)), + longitude_reorder = CircularSort(0, 360), + ## transformation + transform = CDORemapper, + transform_extra_cells = 2, + transform_params = list(grid = 'r360x181', + method = 'conservative', + crop = c(lon.min, lon.max, + lat.min, lat.max)), + transform_vars = c('latitude', 'longitude'), + ## + synonims = list(latitude = c('lat', 'latitude'), + longitude = c('lon', 'longitude')), + return_vars = list(time = 'sdate', + longitude = NULL, + latitude = NULL), + retrieve = TRUE + ) +``` + +4. Run the above script. Check the dimensions and the values of longitude and latitude. +```r +dim(data) + dat var sdate ensemble time latitude longitude + 1 1 1 1 7 181 360 + +longitude <- attr(data, 'Variables')$common$longitude +range(longitude) +[1] 0 359 + +latitude <- attr(data, 'Variables')$common$latitude +range(latitude) +[1] -90 90 +``` + diff --git a/inst/doc/tutorial/PATC2021/nord3_demo.R b/inst/doc/tutorial/PATC2021/nord3_demo.R new file mode 100644 index 0000000000000000000000000000000000000000..af203627babe3129ff0437248c494106b4b0ab56 --- /dev/null +++ b/inst/doc/tutorial/PATC2021/nord3_demo.R @@ -0,0 +1,77 @@ +library(startR) + + repos <- "/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" + var <- 'tas' + sdate <- c('20170101', '20170201') + lon.min <- 0 + lon.max <- 359.9 + lat.min <- -90 + lat.max <- 90 + + data <- Start(dat = repos, + var = var, + sdate = sdate, + ensemble = indices(1:50), + time = 'all', + latitude = values(list(lat.min, lat.max)), + latitude_reorder = Sort(), + longitude = values(list(lon.min, lon.max)), + longitude_reorder = CircularSort(0, 360), + synonims = list(latitude = c('lat', 'latitude'), + longitude = c('lon', 'longitude')), + return_vars = list(time = 'sdate', + longitude = NULL, latitude = NULL), + retrieve = FALSE + ) + + func <- function(x, ens_dim, conf, pval) { + # ensemble mean + ens_mean <- apply(x, which(names(dim(x)) != ens_dim), mean) + # temporal trend + trend <- s2dv:::.Trend(ens_mean, conf = conf, pval = pval)$trend + + return(trend) + } + step <- Step(func, target_dims = c('ensemble', 'time'), + output_dims = list(trend = 'stats'), + use_libraries = c('s2dv')) + wf <- AddStep(data, step, ens_dim = 1, conf = FALSE, pval = FALSE) + +#-------------------user-defined--------------------- + queue_host <- 'nord1' + temp_dir <- '/gpfs/scratch/bsc32/bsc32734/startR_hpc/' + ecflow_suite_dir <- '/home/Earth/aho/startR_local/' +#---------------------------------------------------- + + res <- Compute(wf, + chunks = list(latitude = 2, + longitude = 3), + threads_load = 2, + threads_compute = 4, + cluster = list(queue_host = queue_host, + queue_type = 'lsf', + temp_dir = temp_dir, + cores_per_job = 2, + job_wallclock = '05:00', + max_jobs = 4, + extra_queue_params = list('#BSUB -q bsc_es'), + bidirectional = FALSE, + polling_period = 10 + ), + ecflow_suite_dir = ecflow_suite_dir, + wait = FALSE + ) #$output1 + +# Save the header of res +saveRDS(res, file = '~/nord3_demo_header.Rds') + +# You can leave now. When you come back next time, just read the .Rds file +res_collect <- readRDS('~/nord3_demo_header.Rds') +result <- Collect(res_collect, wait = TRUE) +# Save the result if needed +saveRDS(result, file = '~/nord3_demo_result.Rds') + +# Check the result +str(result) +dim(res$trend) + diff --git a/inst/doc/tutorial/crps.png b/inst/doc/tutorial/internal_training_20200902/crps.png similarity index 100% rename from inst/doc/tutorial/crps.png rename to inst/doc/tutorial/internal_training_20200902/crps.png diff --git a/inst/doc/tutorial/hands-on_part1.md b/inst/doc/tutorial/internal_training_20200902/hands-on_part1.md similarity index 100% rename from inst/doc/tutorial/hands-on_part1.md rename to inst/doc/tutorial/internal_training_20200902/hands-on_part1.md diff --git a/inst/doc/tutorial/hands-on_part1_ans.md b/inst/doc/tutorial/internal_training_20200902/hands-on_part1_ans.md similarity index 100% rename from inst/doc/tutorial/hands-on_part1_ans.md rename to inst/doc/tutorial/internal_training_20200902/hands-on_part1_ans.md diff --git a/inst/doc/tutorial/hands-on_part2.md b/inst/doc/tutorial/internal_training_20200902/hands-on_part2.md similarity index 100% rename from inst/doc/tutorial/hands-on_part2.md rename to inst/doc/tutorial/internal_training_20200902/hands-on_part2.md diff --git a/inst/doc/tutorial/hands-on_part2_ans.md b/inst/doc/tutorial/internal_training_20200902/hands-on_part2_ans.md similarity index 100% rename from inst/doc/tutorial/hands-on_part2_ans.md rename to inst/doc/tutorial/internal_training_20200902/hands-on_part2_ans.md diff --git a/inst/doc/tutorial/nord3_demo.R b/inst/doc/tutorial/internal_training_20200902/nord3_demo.R similarity index 100% rename from inst/doc/tutorial/nord3_demo.R rename to inst/doc/tutorial/internal_training_20200902/nord3_demo.R diff --git a/inst/doc/tutorial/rmsss.png b/inst/doc/tutorial/internal_training_20200902/rmsss.png similarity index 100% rename from inst/doc/tutorial/rmsss.png rename to inst/doc/tutorial/internal_training_20200902/rmsss.png diff --git a/inst/doc/tutorial/startR_tutorial_20200902.pdf b/inst/doc/tutorial/internal_training_20200902/startR_tutorial_20200902.pdf similarity index 100% rename from inst/doc/tutorial/startR_tutorial_20200902.pdf rename to inst/doc/tutorial/internal_training_20200902/startR_tutorial_20200902.pdf