From 59afd3e85b812ae22f25b3412860622903d0e81f Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Tue, 16 Jan 2024 15:30:59 +0100 Subject: [PATCH 01/14] Update README with the use case and s2dv_cube example --- README.md | 54 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 2dfbb5b7..4431be20 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ A part from this GitLab project, that allows you to monitor CSTools progress, to - The CRAN repository [https://CRAN.R-project.org/package=CSTools](https://CRAN.R-project.org/package=CSTools) which includes the user manual and vignettes. - Video tutorials [https://www.medscope-project.eu/products/tool-box/cstools-video-tutorials/](https://www.medscope-project.eu/products/tool-box/cstools-video-tutorials/). - Other resources are under-development such [training material](https://earth.bsc.es/gitlab/external/cstools/-/tree/MEDCOF2022/inst/doc/MEDCOF2022) and a [full reproducible use case for forecast calibration](https://earth.bsc.es/gitlab/external/cstools/-/tree/develop-CalibrationVignette/FOCUS_7_2). +- See [**use cases**](inst/doc/usecase.md) Installation ------------ @@ -58,22 +59,43 @@ This package is designed to be compatible with other R packages such as [s2dv](h The class `s2dv_cube` is mainly a list of named elements to keep data and metadata in a single object. Basic structure of the object: ```r -$ data: [data array] -$ dims: [dimensions vector] -$ coords: [List of coordinates vectors] - $ sdate - $ time - $ lon - [...] -$ attrs: [List of the attributes] - $ Variable: - $ varName - $ metadata - $ Datasets - $ Dates - $ source_files - $ when - $ load_parameters +> lonlat_temp_st$exp +'s2dv_cube' +Data [ 279.994110107422, 280.337463378906, 279.450866699219, 281.992889404297, 280.921813964844, 278.924621582031, 280.738433837891, 279.409515380859 ... ] +Dimensions ( dataset = 1, var = 1, member = 15, sdate = 6, ftime = 3, lat = 22, lon = 53 ) +Coordinates + * dataset : dat1 + * var : tas + member : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 + * sdate : 20001101, 20011101, 20021101, 20031101, 20041101, 20051101 + ftime : 1, 2, 3 + * lat : 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27 + * lon : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359 +Attributes + Dates : 2000-11-01 2001-11-01 2002-11-01 2003-11-01 2004-11-01 ... + varName : tas + metadata : + lat + units : degrees_north + long name : latitude + other : ndims, size, standard_name, axis + lon + units : degrees_east + long name : longitude + other : ndims, size, standard_name, axis + ftime + units : hours since 2000-11-01 00:00:00 + other : ndims, size, standard_name, calendar + tas + units : K + long name : 2 metre temperature + other : prec, dim, unlim, make_missing_value, missval, hasAddOffset, hasScaleFact, code, table + Datasets : dat1 + when : 2023-10-02 10:11:06 + source_files : /esarchive/exp/ecmwf/system5c3s/monthly_mean/tas_f6h/tas_20001101.nc ... + load_parameters : + ( dat1 ) : dataset = dat1, var = tas, sdate = 20001101 ... + ... ``` More information about the `s2dv_cube` object class can be found here: [description of the s2dv_cube object structure document](https://docs.google.com/document/d/1ko37JFl_h6mOjDKM5QSQGikfLBKZq1naL11RkJIwtMM/edit?usp=sharing). -- GitLab From f28fbe9f669dabab30180f3540ba2ad5c3aae4f1 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Tue, 16 Jan 2024 15:34:11 +0100 Subject: [PATCH 02/14] Update README --- README.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/README.md b/README.md index 4431be20..6da47ff3 100644 --- a/README.md +++ b/README.md @@ -100,11 +100,7 @@ Attributes More information about the `s2dv_cube` object class can be found here: [description of the s2dv_cube object structure document](https://docs.google.com/document/d/1ko37JFl_h6mOjDKM5QSQGikfLBKZq1naL11RkJIwtMM/edit?usp=sharing). -The current `s2dv_cube` object (CSTools 5.0.0) differs from the original object used in the previous versions of the packages. If you have **questions** on this change you can follow some of the points below: - -- [New s2dv_cube object discussion](https://earth.bsc.es/gitlab/external/cstools/-/issues/94) -- [How to deal with the compatibility break](https://earth.bsc.es/gitlab/external/cstools/-/issues/112) -- [Testing issue and specifications](https://earth.bsc.es/gitlab/external/cstools/-/issues/110) +The current `s2dv_cube` object (CSTools version > 5.0.0) differs from the original object used in the previous versions of the packages. If you have doubts on this change you can follow some of the issues: [New s2dv_cube object discussion](https://earth.bsc.es/gitlab/external/cstools/-/issues/94), [How to deal with the compatibility break](https://earth.bsc.es/gitlab/external/cstools/-/issues/112) and [Testing issue and specifications](https://earth.bsc.es/gitlab/external/cstools/-/issues/110) Contribute ---------- -- GitLab From dbbeabfca5ec98f1ce99e802b89070ae0a9d0801 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Tue, 16 Jan 2024 15:53:04 +0100 Subject: [PATCH 03/14] Update description of the s2dv_cube in README --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 6da47ff3..46edb088 100644 --- a/README.md +++ b/README.md @@ -47,21 +47,21 @@ Overview The CSTools package functions can be distributed in the following methods: -- **Data retrieval and formatting:** CST_Load, CST_Anomaly, CST_MergeDims, CST_SplitDims, CST_Subset, as.s2dv_cube, s2dv_cube, CST_SaveExp. +- **Data retrieval and formatting:** CST_Start, CST_SaveExp, CST_MergeDims, CST_SplitDim, CST_Subset, CST_InsertDim, CST_ChangeDimNames, as.s2dv_cube and s2dv_cube. - **Classification:** CST_MultiEOF, CST_WeatherRegimes, CST_RegimsAssign, CST_CategoricalEnsCombination, CST_EnsClustering. - **Downscaling:** CST_Analogs, CST_RainFARM, CST_RFTemp, CST_AdamontAnalog, CST_AnalogsPredictors. -- **Correction:** CST_BEI_Weighting, CST_BiasCorrection, CST_Calibration, CST_QuantileMapping, CST_DynBiasCorrection. +- **Correction and transformation:** CST_BEI_Weighting, CST_BiasCorrection, CST_Calibration, CST_QuantileMapping, CST_DynBiasCorrection, CST_Anomaly. - **Assessment:** CST_MultiMetric, CST_MultivarRMSE - **Visualization:** PlotCombinedMap, PlotForecastPDF, PlotMostLikelyQuantileMap, PlotPDFsOLE, PlotTriangles4Categories, PlotWeeklyClim. This package is designed to be compatible with other R packages such as [s2dv](https://CRAN.R-project.org/package=s2dv), [startR](https://CRAN.R-project.org/package=startR), [CSIndicators](https://CRAN.R-project.org/package=CSIndicators), [CSDownscale](https://earth.bsc.es/gitlab/es/csdownscale). Functions with the prefix **CST_** deal with a common object called `s2dv_cube` as inputs. Also, this object can be created from Load (s2dv) and from Start (startR) directly. Multiple functions from different packages can operate on this common data structure to easily define a complete post-processing workflow. -The class `s2dv_cube` is mainly a list of named elements to keep data and metadata in a single object. Basic structure of the object: +An `s2dv_cube` is an object to store ordered multidimensional array with named dimensions, specific coordinates and stored metadata. Its “methods” are the **CST** prefix functions. The basic structure of the class `s2dv_cube` is a list of lists. The first level elements are: `data`, `dims`, `coords` and `attrs`. To access any specific element it will be done using the `$` operator. +As an example, this is how th sample data looks like (`lonlat_temp_st$exp`) ```r -> lonlat_temp_st$exp 's2dv_cube' -Data [ 279.994110107422, 280.337463378906, 279.450866699219, 281.992889404297, 280.921813964844, 278.924621582031, 280.738433837891, 279.409515380859 ... ] +Data [ 279.994110107422, 280.337463378906, 279.450866699219, 281.992889404297, 280.921813964844, ... ] Dimensions ( dataset = 1, var = 1, member = 15, sdate = 6, ftime = 3, lat = 22, lon = 53 ) Coordinates * dataset : dat1 @@ -100,7 +100,7 @@ Attributes More information about the `s2dv_cube` object class can be found here: [description of the s2dv_cube object structure document](https://docs.google.com/document/d/1ko37JFl_h6mOjDKM5QSQGikfLBKZq1naL11RkJIwtMM/edit?usp=sharing). -The current `s2dv_cube` object (CSTools version > 5.0.0) differs from the original object used in the previous versions of the packages. If you have doubts on this change you can follow some of the issues: [New s2dv_cube object discussion](https://earth.bsc.es/gitlab/external/cstools/-/issues/94), [How to deal with the compatibility break](https://earth.bsc.es/gitlab/external/cstools/-/issues/112) and [Testing issue and specifications](https://earth.bsc.es/gitlab/external/cstools/-/issues/110) +**Note:** The current `s2dv_cube` object (CSTools version > 5.0.0) differs from the original object used in the previous versions of the packages. If you have doubts on this change you can follow some of the issues: [New s2dv_cube object discussion](https://earth.bsc.es/gitlab/external/cstools/-/issues/94), [How to deal with the compatibility break](https://earth.bsc.es/gitlab/external/cstools/-/issues/112) and [Testing issue and specifications](https://earth.bsc.es/gitlab/external/cstools/-/issues/110) Contribute ---------- -- GitLab From 031b0451555da0fdc8022e3a1e8d136ecf06b779 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Tue, 16 Jan 2024 15:55:25 +0100 Subject: [PATCH 04/14] Update sentence in README --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 46edb088..581ea145 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ A part from this GitLab project, that allows you to monitor CSTools progress, to - The CRAN repository [https://CRAN.R-project.org/package=CSTools](https://CRAN.R-project.org/package=CSTools) which includes the user manual and vignettes. - Video tutorials [https://www.medscope-project.eu/products/tool-box/cstools-video-tutorials/](https://www.medscope-project.eu/products/tool-box/cstools-video-tutorials/). - Other resources are under-development such [training material](https://earth.bsc.es/gitlab/external/cstools/-/tree/MEDCOF2022/inst/doc/MEDCOF2022) and a [full reproducible use case for forecast calibration](https://earth.bsc.es/gitlab/external/cstools/-/tree/develop-CalibrationVignette/FOCUS_7_2). -- See [**use cases**](inst/doc/usecase.md) +- See and run package [**use cases**](inst/doc/usecase.md) Installation ------------ @@ -54,8 +54,6 @@ The CSTools package functions can be distributed in the following methods: - **Assessment:** CST_MultiMetric, CST_MultivarRMSE - **Visualization:** PlotCombinedMap, PlotForecastPDF, PlotMostLikelyQuantileMap, PlotPDFsOLE, PlotTriangles4Categories, PlotWeeklyClim. -This package is designed to be compatible with other R packages such as [s2dv](https://CRAN.R-project.org/package=s2dv), [startR](https://CRAN.R-project.org/package=startR), [CSIndicators](https://CRAN.R-project.org/package=CSIndicators), [CSDownscale](https://earth.bsc.es/gitlab/es/csdownscale). Functions with the prefix **CST_** deal with a common object called `s2dv_cube` as inputs. Also, this object can be created from Load (s2dv) and from Start (startR) directly. Multiple functions from different packages can operate on this common data structure to easily define a complete post-processing workflow. - An `s2dv_cube` is an object to store ordered multidimensional array with named dimensions, specific coordinates and stored metadata. Its “methods” are the **CST** prefix functions. The basic structure of the class `s2dv_cube` is a list of lists. The first level elements are: `data`, `dims`, `coords` and `attrs`. To access any specific element it will be done using the `$` operator. As an example, this is how th sample data looks like (`lonlat_temp_st$exp`) @@ -98,6 +96,8 @@ Attributes ... ``` +This package is designed to be compatible with other R packages such as [s2dv](https://CRAN.R-project.org/package=s2dv), [startR](https://CRAN.R-project.org/package=startR), [CSIndicators](https://CRAN.R-project.org/package=CSIndicators), [CSDownscale](https://earth.bsc.es/gitlab/es/csdownscale). + More information about the `s2dv_cube` object class can be found here: [description of the s2dv_cube object structure document](https://docs.google.com/document/d/1ko37JFl_h6mOjDKM5QSQGikfLBKZq1naL11RkJIwtMM/edit?usp=sharing). **Note:** The current `s2dv_cube` object (CSTools version > 5.0.0) differs from the original object used in the previous versions of the packages. If you have doubts on this change you can follow some of the issues: [New s2dv_cube object discussion](https://earth.bsc.es/gitlab/external/cstools/-/issues/94), [How to deal with the compatibility break](https://earth.bsc.es/gitlab/external/cstools/-/issues/112) and [Testing issue and specifications](https://earth.bsc.es/gitlab/external/cstools/-/issues/110) -- GitLab From fac6c02a8be5a3d2794e929dade255cd1b43024a Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Tue, 16 Jan 2024 16:52:46 +0100 Subject: [PATCH 05/14] Add subset example and modify names of use case saveexp --- inst/doc/usecase/ex1_create.R | 26 ++++ .../{UseCase4_CST_SaveExp.R => ex2_save.R} | 21 ++- inst/doc/usecase/ex3_modify_dims.R | 26 ++++ inst/doc/usecase/ex4_subset.R | 133 ++++++++++++++++++ 4 files changed, 195 insertions(+), 11 deletions(-) create mode 100644 inst/doc/usecase/ex1_create.R rename inst/doc/usecase/{UseCase4_CST_SaveExp.R => ex2_save.R} (97%) create mode 100644 inst/doc/usecase/ex3_modify_dims.R create mode 100644 inst/doc/usecase/ex4_subset.R diff --git a/inst/doc/usecase/ex1_create.R b/inst/doc/usecase/ex1_create.R new file mode 100644 index 00000000..b1bd9f0e --- /dev/null +++ b/inst/doc/usecase/ex1_create.R @@ -0,0 +1,26 @@ +#******************************************************************************* +# Title: Script to test examples of CST_SaveExp +# Author: Eva Rifà Rovira +# Date: 29/11/2024 +#******************************************************************************* + +#------------------------------------------------------------------------------- +# Needed packages before a new version is installed +library(CSIndicators) +library(multiApply) +library(easyNCDF) +library(s2dv) +library(ClimProjDiags) +library(CSTools) +library(startR) +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_SaveExp.R") +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/zzz.R") + +################################################################################ +#----------------------------------------------------- +# Example 1: Multidimensional array and Dates, without metadata and coordinates +#----------------------------------------------------- +# (1.1) Minimal use case, without Dates + + +################################################################################ \ No newline at end of file diff --git a/inst/doc/usecase/UseCase4_CST_SaveExp.R b/inst/doc/usecase/ex2_save.R similarity index 97% rename from inst/doc/usecase/UseCase4_CST_SaveExp.R rename to inst/doc/usecase/ex2_save.R index 926754f3..ba8da370 100644 --- a/inst/doc/usecase/UseCase4_CST_SaveExp.R +++ b/inst/doc/usecase/ex2_save.R @@ -1,7 +1,7 @@ #******************************************************************************* -# Script to test examples of CST_SaveExp -# Eva Rifà Rovira -# 29/11/2024 +# Title: Script to test examples of CST_SaveExp +# Author: Eva Rifà Rovira +# Date: 29/11/2024 #******************************************************************************* #------------------------------------------------------------------------------- @@ -13,13 +13,12 @@ library(s2dv) library(ClimProjDiags) library(CSTools) library(startR) -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/develop-SaveCube/R/CST_SaveExp.R") -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/develop-SaveCube/R/zzz.R") +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_SaveExp.R") +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/zzz.R") ################################################################################ -# Tests: #----------------------------------------------------- -# Tests 1: Multidimensional array and Dates, without metadata and coordinates +# Example 1: Multidimensional array and Dates, without metadata and coordinates #----------------------------------------------------- # (1.1) Minimal use case, without Dates data <- array(1:5, dim = c(sdate = 5, lon = 4, lat = 4)) @@ -70,7 +69,7 @@ SaveExp(data, ftime_dim = 'ftime', memb_dim = NULL, dat_dim = NULL, extra_string = 'test', global_attrs = list(system = 'tes1', reference = 'test2')) #----------------------------------------------------- -# Tests 2: Test sample data from Start and from Load +# Example 2: Test sample data from Start and from Load #----------------------------------------------------- # (2.1) Test SaveExp @@ -267,7 +266,7 @@ CST_SaveExp(data = data, ftime_dim = NULL, single_file = FALSE, units_hours_since = FALSE) #----------------------------------------------------- -# Test 3: Special cases +# Example 3: Special cases #----------------------------------------------------- # (3.1) Two variables and two datasets in separated files @@ -377,7 +376,7 @@ CST_SaveExp(data = obscube, ftime_dim = 'time', var_dim = 'var', single_file = FALSE, extra_string = 'obs_tas') #----------------------------------------------------- -# Test 4: Time bounds: +# Example 4: Time bounds: #----------------------------------------------------- # example: /esarchive/exp/ncep/cfs-v2/weekly_mean/s2s/tas_f24h/tas_20231128.nc @@ -440,7 +439,7 @@ CST_SaveExp(data = res, ftime_dim = 'time', var_dim = 'var', units_hours_since = FALSE) #----------------------------------------------------- -# Test 5: Read data with Load +# Example 5: Read data with Load #----------------------------------------------------- data <- lonlat_temp$exp diff --git a/inst/doc/usecase/ex3_modify_dims.R b/inst/doc/usecase/ex3_modify_dims.R new file mode 100644 index 00000000..b1bd9f0e --- /dev/null +++ b/inst/doc/usecase/ex3_modify_dims.R @@ -0,0 +1,26 @@ +#******************************************************************************* +# Title: Script to test examples of CST_SaveExp +# Author: Eva Rifà Rovira +# Date: 29/11/2024 +#******************************************************************************* + +#------------------------------------------------------------------------------- +# Needed packages before a new version is installed +library(CSIndicators) +library(multiApply) +library(easyNCDF) +library(s2dv) +library(ClimProjDiags) +library(CSTools) +library(startR) +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_SaveExp.R") +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/zzz.R") + +################################################################################ +#----------------------------------------------------- +# Example 1: Multidimensional array and Dates, without metadata and coordinates +#----------------------------------------------------- +# (1.1) Minimal use case, without Dates + + +################################################################################ \ No newline at end of file diff --git a/inst/doc/usecase/ex4_subset.R b/inst/doc/usecase/ex4_subset.R new file mode 100644 index 00000000..57dadfa5 --- /dev/null +++ b/inst/doc/usecase/ex4_subset.R @@ -0,0 +1,133 @@ +#******************************************************************************* +# Title: Script to test examples of CST_Subset +# Author: Eva Rifà Rovira +# Date: 16/11/2024 +#******************************************************************************* +# This example shows how to subset an array. The basic function to subset +# arrays is Subset, from the package ClimProjDiags. Then, in CSTools, there is +# the CST version of the function to subset all other related information +# of the object 's2dv_cube'. +#------------------------------------------------------------------------------- +# Needed packages before a new version is installed +library(CSTools) +library(ClimProjDiags) +source("https://earth.bsc.es/gitlab/es/ClimProjDiags/-/raw/master/R/Subset.R") +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_Subset.R") + +################################################################################ +#----------------------------------------------------- +# Example 1: Subset an example array +#----------------------------------------------------- +# (1.1) Minimal use case. Spatial coordinates subset. +dat <- array(1:100, dim = c(lat = 10, lon = 10)) +# > dat +# [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] +# [1,] 1 11 21 31 41 51 61 71 81 91 +# [2,] 2 12 22 32 42 52 62 72 82 92 +# [3,] 3 13 23 33 43 53 63 73 83 93 +# [4,] 4 14 24 34 44 54 64 74 84 94 +# [5,] 5 15 25 35 45 55 65 75 85 95 +# [6,] 6 16 26 36 46 56 66 76 86 96 +# [7,] 7 17 27 37 47 57 67 77 87 97 +# [8,] 8 18 28 38 48 58 68 78 88 98 +# [9,] 9 19 29 39 49 59 69 79 89 99 +# [10,] 10 20 30 40 50 60 70 80 90 100 +dat_subset <- Subset(x = dat, along = c('lat', 'lon'), indices = list(1:5, 1:7), + drop = 'all') +# > dat_subset +# [,1] [,2] [,3] [,4] [,5] [,6] [,7] +# [1,] 1 11 21 31 41 51 61 +# [2,] 2 12 22 32 42 52 62 +# [3,] 3 13 23 33 43 53 63 +# [4,] 4 14 24 34 44 54 64 +# [5,] 5 15 25 35 45 55 65 + +#----------------------------------------------------- +# Example 2: Subset an 's2dv_cube' using sample data +#----------------------------------------------------- +# (2.1) We don't want to drop any dimension, select only the first member, +# the first and the second start dates, and also subset the longitude and +# keep only the values from [0, 21]: + +dat <- lonlat_temp_st$exp + +# > dat$dims +# dataset var member sdate ftime lat lon +# 1 1 15 6 3 22 53 + +# > dat +# 's2dv_cube' +# Data [ 279.994110107422, 280.337463378906, 279.450866699219, ... ] +# Dimensions ( dataset = 1, var = 1, member = 15, sdate = 6, ftime = 3, +# lat = 22, lon = 53 ) +# Coordinates +# * dataset : dat1 +# * var : tas +# member : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 +# * sdate : 20001101, 20011101, 20021101, 20031101, 20041101, 20051101 +# ftime : 1, 2, 3 +# * lat : 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, ... +# * lon : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, ... +# Attributes +# Dates : 2000-11-01 2001-11-01 2002-11-01 2003-11-01 2004-11-01 ... +# varName : tas +# metadata : +# lat +# units : degrees_north +# long name : latitude +# lon +# units : degrees_east +# long name : longitude +# ftime +# units : hours since 2000-11-01 00:00:00 +# tas +# units : K +# long name : 2 metre temperature +# Datasets : dat1 +# when : 2023-10-02 10:11:06 +# source_files : /monthly_mean/tas_f6h/tas_20001101.nc ... +# load_parameters : +# ( dat1 ) : dataset = dat1, var = tas, sdate = 20001101 ... +# ... + +dat_subset <- CST_Subset(x = dat, along = c('member', 'sdate', 'lon'), + indices = list(1, 1:2, 1:22), drop = 'none') + +# > dat_subset +# 's2dv_cube' +# Data [ 279.994110107422, 277.161102294922, 278.825836181641, 276.8271484375, 276.052703857422, 276.950805664062, 280.677215576172, 277.285247802734 ... ] +# Dimensions ( dataset = 1, var = 1, member = 1, sdate = 2, ftime = 3, lat = 22, lon = 22 ) +# Coordinates +# * dataset : dat1 +# * var : tas +# member : 1 +# * sdate : 20001101, 20011101 +# ftime : 1, 2, 3 +# * lat : 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27 +# * lon : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21 +# Attributes +# Dates : 2000-11-01 2001-11-01 2000-12-01 2001-12-01 2001-01-01 ... +# varName : tas +# metadata : +# ftime +# units : hours since 2000-11-01 00:00:00 +# other : ndims, size, standard_name, calendar +# lat +# units : degrees_north +# long name : latitude +# other : ndims, size, standard_name, axis +# lon +# units : degrees_east +# long name : longitude +# other : ndims, size, standard_name, axis +# tas +# units : K +# long name : 2 metre temperature +# other : prec, dim, unlim, make_missing_value, missval, hasAddOffset, hasScaleFact, code, table +# Datasets : dat1 +# when : 2023-10-02 10:11:06 +# source_files : /esarchive/exp/ecmwf/system5c3s/monthly_mean/tas_f6h/tas_20001101.nc ... +# load_parameters : +# ( dat1 ) : dataset = dat1, var = tas, sdate = 20001101 ... +# ... +################################################################################ \ No newline at end of file -- GitLab From 2925f1ae282e86d2c4bebe54da54a169246fe562 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Tue, 16 Jan 2024 17:26:06 +0100 Subject: [PATCH 06/14] Correct function namespace of CST_Subset and add use case of creating an 's2dv_cube' --- R/CST_Subset.R | 28 +++++++------- inst/doc/usecase/ex1_create.R | 71 ++++++++++++++++++++++++++++------- 2 files changed, 71 insertions(+), 28 deletions(-) diff --git a/R/CST_Subset.R b/R/CST_Subset.R index 28c8498c..098df3df 100644 --- a/R/CST_Subset.R +++ b/R/CST_Subset.R @@ -79,9 +79,9 @@ CST_Subset <- function(x, along, indices, drop = FALSE, var_dim = NULL, } # Subset data - x$data <- ClimProjDiags::Subset(x$data, along = along, - indices = indices, - drop = drop) + x$data <- Subset(x$data, along = along, + indices = indices, + drop = drop) # Adjust dimensions x$dims <- dim(x$data) # Adjust coordinates @@ -113,10 +113,10 @@ CST_Subset <- function(x, along, indices, drop = FALSE, var_dim = NULL, } if ((!is.null(x$attrs$source_files)) && (dim_name %in% names(dim(x$attrs$source_files)))) { - x$attrs$source_files <- ClimProjDiags::Subset(x$attrs$source_files, - along = dim_name, - indices = index, - drop = drop) + x$attrs$source_files <- Subset(x$attrs$source_files, + along = dim_name, + indices = index, + drop = drop) } } # Remove metadata from variables that were dropped @@ -128,10 +128,10 @@ CST_Subset <- function(x, along, indices, drop = FALSE, var_dim = NULL, if (!(length(time_along) == 0)) { time_indices <- indices[match(time_along, along)] original_dates <- x$attrs$Dates - x$attrs$Dates <- ClimProjDiags::Subset(x$attrs$Dates, - along = time_along, - indices = time_indices, - drop = drop) + x$attrs$Dates <- Subset(x$attrs$Dates, + along = time_along, + indices = time_indices, + drop = drop) } # Subset metadata for (variable in 1:length(names(x$attrs$Variable$metadata))) { @@ -153,9 +153,9 @@ CST_Subset <- function(x, along, indices, drop = FALSE, var_dim = NULL, if (any(is.null(dim(x)), length(dim(x)) == 1)) { l <- x[args_subset[['indices']][[1]]] } else { - l <- ClimProjDiags::Subset(x, along = args_subset[['along']], - indices = args_subset[['indices']], - drop = args_subset[['drop']]) + l <- Subset(x, along = args_subset[['along']], + indices = args_subset[['indices']], + drop = args_subset[['drop']]) } attr.names <- names(attributes(x)) attr.names <- attr.names[attr.names != 'names'] diff --git a/inst/doc/usecase/ex1_create.R b/inst/doc/usecase/ex1_create.R index b1bd9f0e..afe7ece1 100644 --- a/inst/doc/usecase/ex1_create.R +++ b/inst/doc/usecase/ex1_create.R @@ -1,26 +1,69 @@ #******************************************************************************* -# Title: Script to test examples of CST_SaveExp +# Title: Example script to create 's2dv_cube' objects # Author: Eva Rifà Rovira -# Date: 29/11/2024 +# Date: 16/01/2024 #******************************************************************************* - +# This example shows how to create an 's2dv_cube' object. +# There are two ways of creating an 's2dv_cube' object. The first way is +# to use the function s2dv_cube(): create an 's2dv_cube' from scratch with any +# data. In the second example we see the other method with the function +# as.s2dv_cube(). This function is to create an 's2dv_cube' from a +# 'startR_array' or 'load' object. #------------------------------------------------------------------------------- -# Needed packages before a new version is installed -library(CSIndicators) -library(multiApply) -library(easyNCDF) -library(s2dv) -library(ClimProjDiags) +# Needed packages library(CSTools) -library(startR) -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_SaveExp.R") -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/zzz.R") ################################################################################ #----------------------------------------------------- -# Example 1: Multidimensional array and Dates, without metadata and coordinates +# Example 1: Function s2dv_cube() from defined data +#----------------------------------------------------- +# (1.1) Minimal use case, with s2dv_cube function +# We define the array with named dimensions: +dat <- array(1:100, dim = c(time = 10, lat = 4, lon = 10)) +# We define the coordinates as a list of vectors: +coords <- list(time = 1:10, lat = 43:40, lon = 0:9) +# The metadata: +metadata <- list(tas = list(level = '2m'), + lon = list(cdo_grid_name = 'r360x181'), + lat = list(cdo_grid_name = 'r360x181')) +# The creation of Dates array. First the initial date: +ini_date <- as.POSIXct('2010-01-01', format = '%Y-%m-%d') +# The sequence of dates +dates <- seq(ini_date, by = 'days', length.out = 10) +# We define the dates dimensions +dim(dates) <- c(time = 10) +dat_cube <- s2dv_cube(data = dat, coords = coords, + varName = 'tas', metadata = metadata, + Dates = dates, + when = "2019-10-23 19:15:29 CET", + source_files = c("/path/to/file1.nc", "/path/to/file2.nc"), + Datasets = 'test_dataset') +# We print the result: +# > dat_cube +# 's2dv_cube' +# Data [ 1, 2, 3, 4, 5, 6, 7, 8 ... ] +# Dimensions ( time = 10, lat = 4, lon = 10 ) +# Coordinates +# * time : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 +# * lat : 43, 42, 41, 40 +# * lon : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 +# Attributes +# Dates : 2010-01-01 2010-01-02 2010-01-03 2010-01-04 2010-01-05 ... +# varName : tas +# metadata : +# tas +# other : level +# lon +# other : cdo_grid_name +# lat +# other : cdo_grid_name +# Datasets : test_dataset +# when : 2019-10-23 19:15:29 CET +# source_files : /path/to/file1.nc ... + +#----------------------------------------------------- +# Example 2: Function as.s2dv_cube() from 'startR_array' #----------------------------------------------------- -# (1.1) Minimal use case, without Dates ################################################################################ \ No newline at end of file -- GitLab From 02e1d48dc69744f844f33290698b513559eb6842 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Wed, 17 Jan 2024 12:03:06 +0100 Subject: [PATCH 07/14] Add example 1, create an s2dv_cube --- inst/doc/usecase/ex1_create.R | 141 +++++++++++++++++++++++++++++++++- 1 file changed, 138 insertions(+), 3 deletions(-) diff --git a/inst/doc/usecase/ex1_create.R b/inst/doc/usecase/ex1_create.R index afe7ece1..2af830fb 100644 --- a/inst/doc/usecase/ex1_create.R +++ b/inst/doc/usecase/ex1_create.R @@ -7,18 +7,22 @@ # There are two ways of creating an 's2dv_cube' object. The first way is # to use the function s2dv_cube(): create an 's2dv_cube' from scratch with any # data. In the second example we see the other method with the function -# as.s2dv_cube(). This function is to create an 's2dv_cube' from a +# CST_Start(). This function is to create an 's2dv_cube' from a # 'startR_array' or 'load' object. #------------------------------------------------------------------------------- # Needed packages library(CSTools) - +library(startR) ################################################################################ #----------------------------------------------------- # Example 1: Function s2dv_cube() from defined data #----------------------------------------------------- # (1.1) Minimal use case, with s2dv_cube function # We define the array with named dimensions: + +# In this example we use the function s2dv_cube() to create an object of class +# 's2dv_cube' with the correct structure. + dat <- array(1:100, dim = c(time = 10, lat = 4, lon = 10)) # We define the coordinates as a list of vectors: coords <- list(time = 1:10, lat = 43:40, lon = 0:9) @@ -62,8 +66,139 @@ dat_cube <- s2dv_cube(data = dat, coords = coords, # source_files : /path/to/file1.nc ... #----------------------------------------------------- -# Example 2: Function as.s2dv_cube() from 'startR_array' +# Example 2: Function as.s2dv_cube() #----------------------------------------------------- +# (2.1) Using CST_Start + +# For this case, we use a random example. + +# NOTE 1: CST_Start() is just a wrapper of function Start() with the transformation +# to 's2dv_cube' object. +# NOTE 2: In order that the input argument auxiliary functions from startR +# work, we need to call them explicitly the startR namespace. +# (e.g. startR::indices()) + +repos1 <- "/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" +repos2 <- "/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" + +res <- CST_Start(dat = list(list(name = 'system4_m1', path = repos2), + list(name = 'system5_m1', path = repos1)), + var = c('tas', 'sfcWind'), + sdate = c('20160101', '20170101'), + ensemble = startR::indices(1:2), + time = startR::indices(1:2), + lat = startR::indices(1:10), + lon = startR::indices(1:10), + synonims = list(lat = c('lat', 'latitude'), + lon = c('lon', 'longitude')), + return_vars = list(time = 'sdate', + longitude = 'dat', + latitude = 'dat'), + metadata_dims = c('dat', 'var'), + retrieve = TRUE) + + +# Now we can explore the object +# 1st level +names(res) +# "data" "dims" "coords" "attrs" + +dim(res$data) +# dat var sdate ensemble time lat lon +# 2 2 2 2 2 10 10 + +res$coords$lon +# [1] 0.000000 0.703125 1.406250 2.109375 2.812500 3.515625 4.218750 4.921875 +# [9] 5.625000 6.328125 +attr(res$coords$lon, 'indices') +# [1] FALSE +# NOTE: The attribute 'indices' is FALSE, it means that the longitude elements +# are the actual values of longitude coordinate. + +res$coords$ensemble +# [1] 1 2 +# attr(,"indices") +# [1] TRUE +# Now we take a look into the Dates array. It must have the time dimensions +# of the data. +dim(res$attrs$Dates) +# sdate time +# 2 2 + +# To see the nested list structure of the object, we just need to use the +# function str(): +str(res) + +# (2.1) Using as.s2dv_cube() function +# We'll load the data with Start and then we'll transform the 'startR_array' +# to 's2dv_cube' object with the function as.s2dv_cube(). We are going +# to load the same data as before: +repos1 <- "/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" +repos2 <- "/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" + +res <- Start(dat = list(list(name = 'system4_m1', path = repos2), + list(name = 'system5_m1', path = repos1)), + var = c('tas', 'sfcWind'), + sdate = c('20160101', '20170101'), + ensemble = startR::indices(1:2), + time = startR::indices(1:2), + lat = startR::indices(1:10), + lon = startR::indices(1:10), + synonims = list(lat = c('lat', 'latitude'), + lon = c('lon', 'longitude')), + return_vars = list(time = 'sdate', + longitude = 'dat', + latitude = 'dat'), + metadata_dims = c('dat', 'var'), + retrieve = TRUE) + +# Now, we use the function as.s2dv_cube() to transform the 'startR_array' +# into an 's2dv_cube': +res_cube <- as.s2dv_cube(res) +# If we call directly the object directly into the terminal, we can see +# all the elements nicely: + +# > res_cube +# 's2dv_cube' +# Data [ 248.241973876953, 247.365753173828, 6.80753087997437, 5.46453714370728, 247.256896972656, 248.500869750977, 6.25862503051758, 5.76889991760254 ... ] +# Dimensions ( dat = 2, var = 2, sdate = 2, ensemble = 2, time = 2, lat = 10, lon = 10 ) +# Coordinates +# * dat : system4_m1, system5_m1 +# * var : tas, sfcWind +# * sdate : 20160101, 20170101 +# ensemble : 1, 2 +# time : 1, 2 +# * lat : 89.4628215685774, 88.7669513528422, 88.0669716474306, 87.366063433082, 86.6648030134408, 85.9633721608804, 85.2618460607126, 84.5602613830534, 83.8586381286076, 83.1569881285417 +# * lon : 0, 0.703125, 1.40625, 2.109375, 2.8125, 3.515625, 4.21875, 4.921875, 5.625, 6.328125 +# Attributes +# Dates : 2016-02-01 2017-02-01 2016-03-01 2017-03-01 +# varName : tas sfcWind +# metadata : +# time +# units : hours since 2016-01-01 00:00:00 +# other : ndims, size, standard_name, calendar +# lon +# units : degrees_east +# long name : longitude +# other : ndims, size, standard_name, axis +# lat +# units : degrees_north +# long name : latitude +# other : ndims, size, standard_name, axis +# tas +# units : K +# long name : 2 metre temperature +# other : prec, dim, unlim, make_missing_value, missval, hasAddOffset, hasScaleFact, code, table, grid_type +# sfcWind +# units : m s**-1 +# long name : 10 meter windspeed +# other : prec, dim, unlim, make_missing_value, missval, hasAddOffset, hasScaleFact, code, table, grid_type +# Datasets : system4_m1 ... +# when : 2024-01-17 11:38:27 +# source_files : /esarchive/exp/ecmwf/system4_m1/monthly_mean/tas_f6h/tas_20160101.nc ... +# load_parameters : +# ( system4_m1 ) : dat = system4_m1, var = tas ..., sdate = 20160101 ... +# ... ################################################################################ \ No newline at end of file -- GitLab From 8b2f914aa068d5aaa91f5d109f026c2143137622 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Wed, 17 Jan 2024 12:06:28 +0100 Subject: [PATCH 08/14] Add use cases description in file usecase.md --- inst/doc/usecase.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/inst/doc/usecase.md b/inst/doc/usecase.md index 8ebf0478..9b2783a4 100644 --- a/inst/doc/usecase.md +++ b/inst/doc/usecase.md @@ -9,4 +9,8 @@ In this document, you can link to the example scripts for different usage of the 4. [Seasonal forecasts for a river flow](inst/doc/usecase/UseCase3_data_preparation_SCHEME_model.R) 2. **Examples using 's2dv_cube'** - 1. [Save 's2dv_cube'](inst/doc/usecase/UseCase4_SaveExp.R) \ No newline at end of file + 1. [Create an 's2dv_cube'](inst/doc/usecase/ex1_create.R) + 2. [Save 's2dv_cube'](inst/doc/usecase/ex2_save.R) + 3. [Modify any 's2dv_cube' dimension](inst/doc/usecase/ex3_modify_dims.R) + 4. [Subset any 's2dv_cube' dimension](inst/doc/usecase/ex4_subset.R) + \ No newline at end of file -- GitLab From ee957adc4c0fd458f7298d1a40ba107012af1cc1 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Thu, 18 Jan 2024 17:31:44 +0100 Subject: [PATCH 09/14] Add examples 2, 3 and 4; improve CST_MergeDims --- R/CST_MergeDims.R | 26 +++++- inst/doc/usecase.md | 9 +- inst/doc/usecase/ex2_save.R | 2 +- inst/doc/usecase/ex3_modify_dims.R | 137 ++++++++++++++++++++++++++--- inst/doc/usecase/ex4_subset.R | 2 +- 5 files changed, 153 insertions(+), 23 deletions(-) diff --git a/R/CST_MergeDims.R b/R/CST_MergeDims.R index 4b66629e..50bab764 100644 --- a/R/CST_MergeDims.R +++ b/R/CST_MergeDims.R @@ -15,8 +15,6 @@ #' \code{merge_dims} will be used. #'@param na.rm A logical indicating if the NA values should be removed or not. #' -#'@import abind -#'@importFrom ClimProjDiags Subset #'@examples #'data <- 1 : c(2 * 3 * 4 * 5 * 6 * 7) #'dim(data) <- c(time = 7, lat = 2, lon = 3, monthly = 4, member = 6, @@ -35,8 +33,28 @@ CST_MergeDims <- function(data, merge_dims = c('ftime', 'monthly'), if (!inherits(data, 's2dv_cube')) { stop("Parameter 'data' must be of the class 's2dv_cube'.") } + if (is.null(rename_dim)) { + rename_dim <- merge_dims[1] + } + # data data$data <- MergeDims(data$data, merge_dims = merge_dims, rename_dim = rename_dim, na.rm = na.rm) + # dims + data$dims <- dim(data$data) + + # coords + data$coords[merge_dims] <- NULL + data$coords[[rename_dim]] <- 1:dim(data$data)[rename_dim] + attr(data$coords[[rename_dim]], 'indices') <- TRUE + + # attrs + if (all(merge_dims %in% names(dim(data$attrs$Dates)))) { + dim(data$attrs$Dates) <- dim(data$data)[rename_dim] + } else if (any(merge_dims %in% names(dim(data$attrs$Dates)))) { + warning("The dimensions of 'Dates' array will be different from ", + "the temporal dimensions in 'data'. Parameter 'merge_dims' ", + "only includes one temporal dimension of 'Dates'.") + } return(data) } #'Function to Split Dimension @@ -55,12 +73,12 @@ CST_MergeDims <- function(data, merge_dims = c('ftime', 'monthly'), #' \code{merge_dims} will be used. #'@param na.rm A logical indicating if the NA values should be removed or not. #' -#'@import abind -#'@importFrom ClimProjDiags Subset #'@examples #'data <- 1 : 20 #'dim(data) <- c(time = 10, lat = 2) #'new_data <- MergeDims(data, merge_dims = c('time', 'lat')) +#'@import abind +#'@importFrom ClimProjDiags Subset #'@export MergeDims <- function(data, merge_dims = c('time', 'monthly'), rename_dim = NULL, na.rm = FALSE) { diff --git a/inst/doc/usecase.md b/inst/doc/usecase.md index 9b2783a4..272964ab 100644 --- a/inst/doc/usecase.md +++ b/inst/doc/usecase.md @@ -1,16 +1,15 @@ -# Usecase scripts +# Use case and example scripts -In this document, you can link to the example scripts for different usage of the function: +In this document, you will find example scripts of the package. The first ones are use cases of cliimate data assessment. The second ones are example scripts on the use of the 's2dv_cube' object. -1. **Climate data assesment and downscaling** +1. **Use cases of climate data assesment and downscaling** 1. [Bias adjustment for assessment of an extreme event](inst/doc/usecase/UseCase1_WindEvent_March2018.R) 2. [Precipitation Downscaling with RainFARM RF 4](inst/doc/usecase/UseCase2_PrecipitationDownscaling_RainFARM_RF4.R) 3. [Precipitation Downscaling with RainFARM RF 100](inst/doc/usecase/UseCase2_PrecipitationDownscaling_RainFARM_RF100.R) 4. [Seasonal forecasts for a river flow](inst/doc/usecase/UseCase3_data_preparation_SCHEME_model.R) -2. **Examples using 's2dv_cube'** +2. **Examples on how to use 's2dv_cube'** 1. [Create an 's2dv_cube'](inst/doc/usecase/ex1_create.R) 2. [Save 's2dv_cube'](inst/doc/usecase/ex2_save.R) 3. [Modify any 's2dv_cube' dimension](inst/doc/usecase/ex3_modify_dims.R) 4. [Subset any 's2dv_cube' dimension](inst/doc/usecase/ex4_subset.R) - \ No newline at end of file diff --git a/inst/doc/usecase/ex2_save.R b/inst/doc/usecase/ex2_save.R index ba8da370..0f8235c2 100644 --- a/inst/doc/usecase/ex2_save.R +++ b/inst/doc/usecase/ex2_save.R @@ -1,5 +1,5 @@ #******************************************************************************* -# Title: Script to test examples of CST_SaveExp +# Title: Example script to save 's2dv_cube' to NetCDF using CST_SaveExp # Author: Eva Rifà Rovira # Date: 29/11/2024 #******************************************************************************* diff --git a/inst/doc/usecase/ex3_modify_dims.R b/inst/doc/usecase/ex3_modify_dims.R index b1bd9f0e..6db1d2b6 100644 --- a/inst/doc/usecase/ex3_modify_dims.R +++ b/inst/doc/usecase/ex3_modify_dims.R @@ -1,26 +1,139 @@ #******************************************************************************* -# Title: Script to test examples of CST_SaveExp +# Title: Script to modify the dimensions of the 's2dv_cube' # Author: Eva Rifà Rovira -# Date: 29/11/2024 +# Date: 18/01/2024 #******************************************************************************* #------------------------------------------------------------------------------- # Needed packages before a new version is installed -library(CSIndicators) -library(multiApply) -library(easyNCDF) -library(s2dv) -library(ClimProjDiags) library(CSTools) -library(startR) -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_SaveExp.R") -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/zzz.R") +source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_ChangeDimNames.R") ################################################################################ +# NOTE: for all the examples, we are going to use the sample data. #----------------------------------------------------- -# Example 1: Multidimensional array and Dates, without metadata and coordinates +# Example 1: Change dimension names with CST_ChangeDimNames #----------------------------------------------------- -# (1.1) Minimal use case, without Dates +# With using this function, we can change the dimension names in all elements +# of the 's2dv_cube' object: +# Example with sample data: +# Check original dimensions and coordinates +lonlat_temp$exp$dims +names(lonlat_temp$exp$coords) +dim(lonlat_temp$exp$attrs$Dates) +# Change 'dataset' to 'dat' and 'ftime' to 'time' +exp <- CST_ChangeDimNames(lonlat_temp$exp, + original_names = c("dataset", "ftime", "lon", "lat"), + new_names = c("dat", "time", "longitude", "latitude")) +# Check new dimensions and coordinates +exp$dims +names(exp$coords) +dim(exp$attrs$Dates) +#----------------------------------------------------- +# Example 2: Insert a new dimension with CST_InsertDim +#----------------------------------------------------- +# With this function, we can add a dimension into the 's2dv_cube'. +# When the dimension that we want to add has lenght greater than 1, the +# values of the data are repeated for that new dimension. + +# Example with sample data: +# Check original dimensions and coordinates +lonlat_temp$exp$dims +names(lonlat_temp$exp$coords) +# Add 'variable' dimension +exp <- CST_InsertDim(lonlat_temp$exp, + posdim = 2, + lendim = 2, + name = "variable", + values = c("tas", "tos")) +# Check new dimensions and coordinates +exp$dims +exp$coords$variable +# We see that the values will be repeated along the new dimension: +exp$data[, , 1, 1, 1, 1, 1] + +#----------------------------------------------------- +# Example 3: Merge two dimensions with CST_MergeDims +#----------------------------------------------------- +# In this example, we will merge the dimensions corresponding to the latitude +# and the longitude of the data. The new dimension will be named 'grid'. +new_data <- CST_MergeDims(lonlat_temp$exp, merge_dims = c('lat', 'lon'), + rename_dim = 'grid') +dim(new_data$data) +# dataset member sdate ftime grid +# 1 15 6 3 1166 +names(new_data$coords) +# [1] "dataset" "member" "sdate" "ftime" "grid" +new_data +# NOTE: Be aware that when we print the object, we see that its name in +# "Coordinates" field appears without the asterisk (*) at its left. This means +# that the values of that coordinate, are indices, not the actual values. We +# can also find this information with the attribute "indices": +attributes(new_data$coords$grid) +# $indices +# [1] TRUE + +# Now, we want to merge time dimensions start date and forecast time: +new_data <- CST_MergeDims(data = lonlat_temp_st$exp, merge_dims = c('sdate', 'ftime')) +# In this case, the Dates dimensions will be merged too. +dim(new_data$attrs$Dates) +# sdate +# 18 + +# However, when we want to merge temporal and other dimensions nature, +# the Dates dimensions are kept as the original. In this case, the function +# returns a Warning Message, we must pay attention! +new_data <- CST_MergeDims(data = lonlat_temp$exp, + merge_dims = c('lat', 'ftime'), + rename_dim = 'newdim') + +#----------------------------------------------------- +# Example 4: Split two dimensions with CST_SplitDim +#----------------------------------------------------- +# In this example, we will start working first with the function SplitDim, +# that it can be used to split dimensions of an array. + +# NOTE: Take into account that time dimensions will be treated differently than +# other dimensions: + +# Decadal example: We define an array of consecutive days of different years: +dates <- seq(as.Date("01-01-2000", "%d-%m-%Y", tz = 'UTC'), + as.Date("31-12-2005","%d-%m-%Y", tz = 'UTC'), "day") +dim(dates) <- c(time = 2192) + +# Now, we will split the array in a new 'year' dimension: +dates_year <- SplitDim(dates, indices = dates, + split_dim = 'time', freq = 'year') +# time year +# 366 6 +# Now, we can try: freq = 'month' and 'day' +dates_month <- SplitDim(dates, indices = dates, + split_dim = 'time', freq = 'month') + +dates_day <- SplitDim(dates, indices = dates, + split_dim = 'time', freq = 'day') +dates <- as.POSIXct(dates * 24 * 3600, origin = '1970-01-01', tz = 'UTC') + +# In the following example, we will use the sample data of the package. We +# will use lonlat_prec_st because it is daily data: + +# NOTE: By Jan 2024, a development is needed regarding updates in other fields +# of the 's2dv_cube' +data_day <- CST_SplitDim(lonlat_prec_st, indices = lonlat_prec_st$attrs$Dates[1,], + split_dim = 'ftime', freq = 'day') +dim(data_day$data) +# dataset var member sdate ftime lat lon day +# 1 1 6 3 1 4 4 31 +data_month <- CST_SplitDim(lonlat_prec_st, indices = lonlat_prec_st$attrs$Dates[1,], + split_dim = 'ftime', freq = 'month') +dim(data_month$data) +# dataset var member sdate ftime lat lon month +# 1 1 6 3 31 4 4 1 +data_year <- CST_SplitDim(lonlat_prec_st, indices = lonlat_prec_st$attrs$Dates[,1], + split_dim = 'sdate', freq = 'year') +dim(data_year$data) +# dataset var member sdate ftime lat lon year +# 1 1 6 1 31 4 4 3 ################################################################################ \ No newline at end of file diff --git a/inst/doc/usecase/ex4_subset.R b/inst/doc/usecase/ex4_subset.R index 57dadfa5..d86b227e 100644 --- a/inst/doc/usecase/ex4_subset.R +++ b/inst/doc/usecase/ex4_subset.R @@ -1,5 +1,5 @@ #******************************************************************************* -# Title: Script to test examples of CST_Subset +# Title: Example script to subset any dimension of an 's2dv_cube' # Author: Eva Rifà Rovira # Date: 16/11/2024 #******************************************************************************* -- GitLab From cd81c935c6ed9b02e6e5a9b97559535e967b5c24 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Fri, 19 Jan 2024 11:28:11 +0100 Subject: [PATCH 10/14] Correct unit test MergeDims --- R/CST_MergeDims.R | 6 +- tests/testthat/test-CST_MergeDims.R | 111 ++++++++++++++++++---------- 2 files changed, 76 insertions(+), 41 deletions(-) diff --git a/R/CST_MergeDims.R b/R/CST_MergeDims.R index 50bab764..dabdc57f 100644 --- a/R/CST_MergeDims.R +++ b/R/CST_MergeDims.R @@ -42,11 +42,15 @@ CST_MergeDims <- function(data, merge_dims = c('ftime', 'monthly'), # dims data$dims <- dim(data$data) + # rename_dim + if (length(rename_dim) > 1) { + rename_dim <- as.character(rename_dim[1]) + } # coords data$coords[merge_dims] <- NULL data$coords[[rename_dim]] <- 1:dim(data$data)[rename_dim] attr(data$coords[[rename_dim]], 'indices') <- TRUE - + # attrs if (all(merge_dims %in% names(dim(data$attrs$Dates)))) { dim(data$attrs$Dates) <- dim(data$data)[rename_dim] diff --git a/tests/testthat/test-CST_MergeDims.R b/tests/testthat/test-CST_MergeDims.R index f7eac6ac..a99d5eba 100644 --- a/tests/testthat/test-CST_MergeDims.R +++ b/tests/testthat/test-CST_MergeDims.R @@ -1,65 +1,96 @@ ############################################## -test_that("Sanity checks", { +# data1 +data1 <- list(data = 1:10) +class(data1) <- 's2dv_cube' + +# data2 +data <- 1 : 20 +dim(data) <- c(time = 20) +data2 <- list(data = data) +data2$dims <- dim(data) +data2$coords <- list(time = 1:20) +attr(data2$coords$time, 'indices') <- TRUE +class(data2) <- 's2dv_cube' + +# exp +exp <- 1 : 20 +dim(exp) <- c(time = 10, lat = 2) +exp <- list(data = exp) +class(exp) <- 's2dv_cube' + +# data3 +data3 <- data2 +names(dim(data3$data)) <- 'Dim1' +data3$dims <- dim(data3$data) +names(data3$coords) <- 'Dim1' + +############################################## + +test_that("1. Sanity checks", { expect_error( CST_MergeDims(data = 1), - paste0("Parameter 'data' must be of the class 's2dv_cube'.")) -data <- list(data = 1:10) -class(data) <- 's2dv_cube' + paste0("Parameter 'data' must be of the class 's2dv_cube'.") + ) expect_error( - CST_MergeDims(data = data), - paste0("Parameter 'data' must have dimensions.")) + CST_MergeDims(data = data1), + paste0("Parameter 'data' must have dimensions.") + ) - data <- 1 : 20 - dim(data) <- c(time = 20) - data <- list(data = data) - class(data) <- 's2dv_cube' expect_error( - CST_MergeDims(data = data), - "Parameter 'merge_dims' must match with dimension names in parameter 'data'.") + CST_MergeDims(data = data2), + "Parameter 'merge_dims' must match with dimension names in parameter 'data'." + ) expect_error( - CST_MergeDims(data = data, merge_dims = 1), - paste0("Parameter 'merge_dims' must be a character vector indicating the names", - " of the dimensions to be merged.")) + CST_MergeDims(data = data2, merge_dims = 1), + paste0("Parameter 'merge_dims' must be a character vector indicating the ", + "names of the dimensions to be merged.") + ) expect_error( - CST_MergeDims(data = data, merge_dims = 'time'), - "Parameter 'merge_dims' must be of length two.") + CST_MergeDims(data = data2, merge_dims = 'time'), + "Parameter 'merge_dims' must be of length two." + ) expect_error( - CST_MergeDims(data = data, merge_dims = c('time', 'sdates')), + CST_MergeDims(data = data2, merge_dims = c('time', 'sdates')), paste0("Parameter 'merge_dims' must match with dimension ", - "names in parameter 'data'.")) + "names in parameter 'data'.") + ) +}) + +############################################## - exp <- 1 : 20 - dim(exp) <- c(time = 10, lat = 2) - exp <- list(data = exp) - class(exp) <- 's2dv_cube' +test_that("2. Output checks", { expect_equal( - CST_MergeDims(data = exp, merge_dims = c('time', 'lat')), data) - + CST_MergeDims(data = exp, merge_dims = c('time', 'lat')), + data2 + ) expect_warning( CST_MergeDims(data = exp, merge_dims = c('time', 'lat', 'lon')), paste0("Only two dimensions can be merge, only the first two dimension", " will be used. To merge further dimensions consider to use this ", - "function multiple times.")) + "function multiple times.") + ) expect_warning( - CST_MergeDims(data = exp, merge_dims = c('time', 'lat'), rename_dim = c('lat', 'lon')), + CST_MergeDims(data = exp, merge_dims = c('time', 'lat'), + rename_dim = c('lat', 'lon')), paste0("Parameter 'rename_dim' has length greater than 1 and only the ", - "first element will be used.")) - names(dim(data$data)) <- 'Dim1' + "first element will be used.") + ) expect_equal( - CST_MergeDims(data = exp, merge_dims = c('time', 'lat'), rename_dim = 'Dim1'), - data) - + CST_MergeDims(data = exp, merge_dims = c('time', 'lat'), + rename_dim = 'Dim1'), + data3 + ) expect_equal( CST_MergeDims(data = exp, merge_dims = c('time', 'lat'), - rename_dim = 'Dim1', na.rm = TRUE), data) - - exp$data[1,] <- NA - data <- c(2 : 10, 12 : 20) - dim(data) <- c(Dim1 = 18) - data <- list(data = data) - class(data) <- 's2dv_cube' + rename_dim = 'Dim1', na.rm = TRUE), + data3 + ) expect_equal( CST_MergeDims(data = exp, merge_dims = c('time', 'lat'), - rename_dim = 'Dim1', na.rm = TRUE), data) + rename_dim = 'Dim1', na.rm = TRUE), + data3 + ) }) + +############################################## \ No newline at end of file -- GitLab From 8c21208ddda8adb0a644e698dc3bac3c95d1ecee Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Fri, 19 Jan 2024 11:56:07 +0100 Subject: [PATCH 11/14] Improve readability of 's2dv_cube' example in README --- README.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 581ea145..24fb07e6 100644 --- a/README.md +++ b/README.md @@ -54,12 +54,12 @@ The CSTools package functions can be distributed in the following methods: - **Assessment:** CST_MultiMetric, CST_MultivarRMSE - **Visualization:** PlotCombinedMap, PlotForecastPDF, PlotMostLikelyQuantileMap, PlotPDFsOLE, PlotTriangles4Categories, PlotWeeklyClim. -An `s2dv_cube` is an object to store ordered multidimensional array with named dimensions, specific coordinates and stored metadata. Its “methods” are the **CST** prefix functions. The basic structure of the class `s2dv_cube` is a list of lists. The first level elements are: `data`, `dims`, `coords` and `attrs`. To access any specific element it will be done using the `$` operator. +An `s2dv_cube` is an object to store ordered multidimensional array with named dimensions, specific coordinates and stored metadata (in-memory representation of a NetCDF file). Its “methods” are the **CST** prefix functions. The basic structure of the class `s2dv_cube` is a list of lists. The first level elements are: `data`, `dims`, `coords` and `attrs`. To access any specific element it will be done using the `$` operator. -As an example, this is how th sample data looks like (`lonlat_temp_st$exp`) +As an example, this is how an `s2dv_cube` looks like (see `lonlat_temp_st$exp`): ```r 's2dv_cube' -Data [ 279.994110107422, 280.337463378906, 279.450866699219, 281.992889404297, 280.921813964844, ... ] +Data [ 279.99, 280.34, 279.45, 281.99, 280.92, ... ] Dimensions ( dataset = 1, var = 1, member = 15, sdate = 6, ftime = 3, lat = 22, lon = 53 ) Coordinates * dataset : dat1 @@ -67,8 +67,8 @@ Coordinates member : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 * sdate : 20001101, 20011101, 20021101, 20031101, 20041101, 20051101 ftime : 1, 2, 3 - * lat : 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27 - * lon : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359 + * lat : 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, ... + * lon : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, ... Attributes Dates : 2000-11-01 2001-11-01 2002-11-01 2003-11-01 2004-11-01 ... varName : tas @@ -76,21 +76,17 @@ Attributes lat units : degrees_north long name : latitude - other : ndims, size, standard_name, axis lon units : degrees_east long name : longitude - other : ndims, size, standard_name, axis ftime units : hours since 2000-11-01 00:00:00 - other : ndims, size, standard_name, calendar tas units : K long name : 2 metre temperature - other : prec, dim, unlim, make_missing_value, missval, hasAddOffset, hasScaleFact, code, table Datasets : dat1 when : 2023-10-02 10:11:06 - source_files : /esarchive/exp/ecmwf/system5c3s/monthly_mean/tas_f6h/tas_20001101.nc ... + source_files : "/ecmwf/system5c3s/monthly_mean/tas_f6h/tas_20001101.nc" ... load_parameters : ( dat1 ) : dataset = dat1, var = tas, sdate = 20001101 ... ... -- GitLab From 102a75587d31dc9a295ac4ea39c785fd11a5cb63 Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Fri, 19 Jan 2024 12:02:57 +0100 Subject: [PATCH 12/14] Improve README --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index 24fb07e6..7b9df56d 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ Overview The CSTools package functions can be distributed in the following methods: -- **Data retrieval and formatting:** CST_Start, CST_SaveExp, CST_MergeDims, CST_SplitDim, CST_Subset, CST_InsertDim, CST_ChangeDimNames, as.s2dv_cube and s2dv_cube. +- **Data retrieval and formatting:** [CST_Start](R/CST_Start.R), [CST_SaveExp](R/CST_SaveExp.R), [CST_MergeDims](R/CST_MergeDims.R), CST_SplitDim, CST_Subset, CST_InsertDim, CST_ChangeDimNames, as.s2dv_cube and s2dv_cube. - **Classification:** CST_MultiEOF, CST_WeatherRegimes, CST_RegimsAssign, CST_CategoricalEnsCombination, CST_EnsClustering. - **Downscaling:** CST_Analogs, CST_RainFARM, CST_RFTemp, CST_AdamontAnalog, CST_AnalogsPredictors. - **Correction and transformation:** CST_BEI_Weighting, CST_BiasCorrection, CST_Calibration, CST_QuantileMapping, CST_DynBiasCorrection, CST_Anomaly. @@ -89,7 +89,6 @@ Attributes source_files : "/ecmwf/system5c3s/monthly_mean/tas_f6h/tas_20001101.nc" ... load_parameters : ( dat1 ) : dataset = dat1, var = tas, sdate = 20001101 ... - ... ``` This package is designed to be compatible with other R packages such as [s2dv](https://CRAN.R-project.org/package=s2dv), [startR](https://CRAN.R-project.org/package=startR), [CSIndicators](https://CRAN.R-project.org/package=CSIndicators), [CSDownscale](https://earth.bsc.es/gitlab/es/csdownscale). -- GitLab From f3c2adef7c1fbd1ef35118fa764a1187e34133ce Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Fri, 19 Jan 2024 12:12:14 +0100 Subject: [PATCH 13/14] Add function links in README --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 7b9df56d..76c82d59 100644 --- a/README.md +++ b/README.md @@ -47,12 +47,12 @@ Overview The CSTools package functions can be distributed in the following methods: -- **Data retrieval and formatting:** [CST_Start](R/CST_Start.R), [CST_SaveExp](R/CST_SaveExp.R), [CST_MergeDims](R/CST_MergeDims.R), CST_SplitDim, CST_Subset, CST_InsertDim, CST_ChangeDimNames, as.s2dv_cube and s2dv_cube. -- **Classification:** CST_MultiEOF, CST_WeatherRegimes, CST_RegimsAssign, CST_CategoricalEnsCombination, CST_EnsClustering. -- **Downscaling:** CST_Analogs, CST_RainFARM, CST_RFTemp, CST_AdamontAnalog, CST_AnalogsPredictors. -- **Correction and transformation:** CST_BEI_Weighting, CST_BiasCorrection, CST_Calibration, CST_QuantileMapping, CST_DynBiasCorrection, CST_Anomaly. -- **Assessment:** CST_MultiMetric, CST_MultivarRMSE -- **Visualization:** PlotCombinedMap, PlotForecastPDF, PlotMostLikelyQuantileMap, PlotPDFsOLE, PlotTriangles4Categories, PlotWeeklyClim. +- **Data retrieval and formatting:** [CST_Start](R/CST_Start.R), [CST_SaveExp](R/CST_SaveExp.R), [CST_MergeDims](R/CST_MergeDims.R), [CST_SplitDim](R/CST_SplitDim.R), [CST_Subset](R/CST_Subset), [CST_InsertDim](R/CST_InsertDim.R), [CST_ChangeDimNames](R/CST_ChangeDimNames.R), [as.s2dv_cube](R/as.s2dv_cube.R) and [s2dv_cube](R/s2dv_cube.R). +- **Classification:** [CST_MultiEOF](R/CST_MultiEOF.R), [CST_WeatherRegimes](R/CST_WeatherRegimes.R), [CST_RegimsAssign](R/CST_RegimesAssign.R), [CST_CategoricalEnsCombination](R/CST_CategoricalEnsCombination.R), [CST_EnsClustering](R/CST_EnsClustering.R). +- **Downscaling:** [CST_Analogs](R/CST_Analogs.R), [CST_RainFARM](R/CST_RainFARM.R), [CST_RFTemp](R/CST_RFTemp.R), [CST_AdamontAnalog](R/CST_AdamontAnalog.R), [CST_AnalogsPredictors](R/CST_AnalogsPredictors.R). +- **Correction and transformation:** [CST_BiasCorrection](R/CST_BiasCorrection.R), [CST_Calibration](R/CST_Calibration.R), [CST_QuantileMapping](R/CST_QuantileMapping.R), [CST_Anomaly](R/CST_Anomaly.R), [CST_BEI_Weighting](R/CST_BEI_Weighting.R), [CST_DynBiasCorrection](R/CST_DynBiasCorrection.R). +- **Assessment:** [CST_MultiMetric](R/CST_MultiMetric.R), [CST_MultivarRMSE](R/CST_MultivarRMSE.R) +- **Visualization:** [PlotCombinedMap](R/PlotCombinedMap.R), [PlotForecastPDF](R/PlotForecastPDF.R), [PlotMostLikelyQuantileMap](R/PlotMostLikelyQuantileMap.R), [PlotPDFsOLE](R/PlotPDFsOLE.R), [PlotTriangles4Categories](R/PlotTriangles4Categories.R), [PlotWeeklyClim](R/PlotWeeklyClim.R). An `s2dv_cube` is an object to store ordered multidimensional array with named dimensions, specific coordinates and stored metadata (in-memory representation of a NetCDF file). Its “methods” are the **CST** prefix functions. The basic structure of the class `s2dv_cube` is a list of lists. The first level elements are: `data`, `dims`, `coords` and `attrs`. To access any specific element it will be done using the `$` operator. -- GitLab From b49bee5121972cd629efc5383afd8fac3b6e852e Mon Sep 17 00:00:00 2001 From: Eva Rifa Date: Fri, 19 Jan 2024 14:11:32 +0100 Subject: [PATCH 14/14] Improve text of 's2dv_cube' operations examples --- inst/doc/usecase/ex1_create.R | 50 +++++++++++--------- inst/doc/usecase/ex2_save.R | 16 +++---- inst/doc/usecase/ex3_modify_dims.R | 74 ++++++++++++++++++++---------- inst/doc/usecase/ex4_subset.R | 43 +++++++++-------- 4 files changed, 111 insertions(+), 72 deletions(-) diff --git a/inst/doc/usecase/ex1_create.R b/inst/doc/usecase/ex1_create.R index 2af830fb..3c07db41 100644 --- a/inst/doc/usecase/ex1_create.R +++ b/inst/doc/usecase/ex1_create.R @@ -4,12 +4,11 @@ # Date: 16/01/2024 #******************************************************************************* # This example shows how to create an 's2dv_cube' object. -# There are two ways of creating an 's2dv_cube' object. The first way is -# to use the function s2dv_cube(): create an 's2dv_cube' from scratch with any -# data. In the second example we see the other method with the function -# CST_Start(). This function is to create an 's2dv_cube' from a -# 'startR_array' or 'load' object. -#------------------------------------------------------------------------------- +# There are two ways of creating an 's2dv_cube' object. +# (1) With the function s2dv_cube(): create it from scratch with any data. +# (2) With the function CST_Start(). This function returns an 's2dv_cube' +# from an 'startR_array'. + # Needed packages library(CSTools) library(startR) @@ -17,32 +16,35 @@ library(startR) #----------------------------------------------------- # Example 1: Function s2dv_cube() from defined data #----------------------------------------------------- -# (1.1) Minimal use case, with s2dv_cube function -# We define the array with named dimensions: +# Minimal use case, with s2dv_cube function. # In this example we use the function s2dv_cube() to create an object of class # 's2dv_cube' with the correct structure. +# (1) We define the array with named dimensions: dat <- array(1:100, dim = c(time = 10, lat = 4, lon = 10)) -# We define the coordinates as a list of vectors: +# (2) We define the coordinates as a list of vectors: coords <- list(time = 1:10, lat = 43:40, lon = 0:9) -# The metadata: +# (3) The metadata: metadata <- list(tas = list(level = '2m'), lon = list(cdo_grid_name = 'r360x181'), lat = list(cdo_grid_name = 'r360x181')) -# The creation of Dates array. First the initial date: +# (4) The creation of Dates array. +# First the initial date: ini_date <- as.POSIXct('2010-01-01', format = '%Y-%m-%d') # The sequence of dates dates <- seq(ini_date, by = 'days', length.out = 10) # We define the dates dimensions dim(dates) <- c(time = 10) +# (5) We call the function s2dv_cube() dat_cube <- s2dv_cube(data = dat, coords = coords, varName = 'tas', metadata = metadata, Dates = dates, when = "2019-10-23 19:15:29 CET", source_files = c("/path/to/file1.nc", "/path/to/file2.nc"), - Datasets = 'test_dataset') -# We print the result: + Datasets = 'test_dataset') + +# We print the result to see the 's2dv_cube' structure: # > dat_cube # 's2dv_cube' # Data [ 1, 2, 3, 4, 5, 6, 7, 8 ... ] @@ -68,16 +70,16 @@ dat_cube <- s2dv_cube(data = dat, coords = coords, #----------------------------------------------------- # Example 2: Function as.s2dv_cube() #----------------------------------------------------- -# (2.1) Using CST_Start - -# For this case, we use a random example. +# (1) Example using CST_Start -# NOTE 1: CST_Start() is just a wrapper of function Start() with the transformation -# to 's2dv_cube' object. +# NOTE 1: CST_Start() is just a wrapper of function Start() with the +# transformation to 's2dv_cube' object. # NOTE 2: In order that the input argument auxiliary functions from startR # work, we need to call them explicitly the startR namespace. # (e.g. startR::indices()) +# We just need to define a CST_Start call with all the information: + repos1 <- "/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" repos2 <- "/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" @@ -98,7 +100,8 @@ res <- CST_Start(dat = list(list(name = 'system4_m1', path = repos2), retrieve = TRUE) -# Now we can explore the object +# Now we can explore the object: + # 1st level names(res) # "data" "dims" "coords" "attrs" @@ -130,10 +133,14 @@ dim(res$attrs$Dates) # function str(): str(res) -# (2.1) Using as.s2dv_cube() function +#----------------------------------------------------- + +# (2) Example using as.s2dv_cube() function + # We'll load the data with Start and then we'll transform the 'startR_array' # to 's2dv_cube' object with the function as.s2dv_cube(). We are going -# to load the same data as before: +# to load the same data as before, with the same call: + repos1 <- "/esarchive/exp/ecmwf/system5_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" repos2 <- "/esarchive/exp/ecmwf/system4_m1/monthly_mean/$var$_f6h/$var$_$sdate$.nc" @@ -156,6 +163,7 @@ res <- Start(dat = list(list(name = 'system4_m1', path = repos2), # Now, we use the function as.s2dv_cube() to transform the 'startR_array' # into an 's2dv_cube': res_cube <- as.s2dv_cube(res) + # If we call directly the object directly into the terminal, we can see # all the elements nicely: diff --git a/inst/doc/usecase/ex2_save.R b/inst/doc/usecase/ex2_save.R index 0f8235c2..7fedd349 100644 --- a/inst/doc/usecase/ex2_save.R +++ b/inst/doc/usecase/ex2_save.R @@ -3,18 +3,14 @@ # Author: Eva Rifà Rovira # Date: 29/11/2024 #******************************************************************************* +# In this script, we'll see multiple ways to store the 's2dv_cube' (CST_SaveExp) +# or the multidimensional array (SaveExp) to NetCDF. -#------------------------------------------------------------------------------- -# Needed packages before a new version is installed +# Needed packages: +library(CSTools) library(CSIndicators) -library(multiApply) -library(easyNCDF) library(s2dv) -library(ClimProjDiags) -library(CSTools) library(startR) -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_SaveExp.R") -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/zzz.R") ################################################################################ #----------------------------------------------------- @@ -461,5 +457,7 @@ CST_SaveExp(data = data, ftime_dim = 'ftime', # latmin = 27, latmax = 48, # lonmin = -12, lonmax = 40, # output = "lonlat") -# Error + +# NOTE: This case hasn't been developed since the function to load data +# that will be maintianed will be CST_Start. ################################################################################ \ No newline at end of file diff --git a/inst/doc/usecase/ex3_modify_dims.R b/inst/doc/usecase/ex3_modify_dims.R index 6db1d2b6..6d1fa5b8 100644 --- a/inst/doc/usecase/ex3_modify_dims.R +++ b/inst/doc/usecase/ex3_modify_dims.R @@ -3,30 +3,32 @@ # Author: Eva Rifà Rovira # Date: 18/01/2024 #******************************************************************************* +# In this example, we will explore different methods to modify the dimensions +# of the 's2dv_cube': +# (1) Changing dimension names +# (2) Adding new dimensions +# (3) Merge 2 dimensions +# (4) Split a dimension -#------------------------------------------------------------------------------- -# Needed packages before a new version is installed +# Needed packages: library(CSTools) -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_ChangeDimNames.R") ################################################################################ -# NOTE: for all the examples, we are going to use the sample data. #----------------------------------------------------- # Example 1: Change dimension names with CST_ChangeDimNames #----------------------------------------------------- # With using this function, we can change the dimension names in all elements # of the 's2dv_cube' object: -# Example with sample data: -# Check original dimensions and coordinates +# (1) Check original dimensions and coordinates lonlat_temp$exp$dims names(lonlat_temp$exp$coords) dim(lonlat_temp$exp$attrs$Dates) -# Change 'dataset' to 'dat' and 'ftime' to 'time' +# (2) Change 'dataset' to 'dat' and 'ftime' to 'time' exp <- CST_ChangeDimNames(lonlat_temp$exp, original_names = c("dataset", "ftime", "lon", "lat"), new_names = c("dat", "time", "longitude", "latitude")) -# Check new dimensions and coordinates +# (3) Check new dimensions and coordinates exp$dims names(exp$coords) dim(exp$attrs$Dates) @@ -35,20 +37,19 @@ dim(exp$attrs$Dates) # Example 2: Insert a new dimension with CST_InsertDim #----------------------------------------------------- # With this function, we can add a dimension into the 's2dv_cube'. -# When the dimension that we want to add has lenght greater than 1, the -# values of the data are repeated for that new dimension. +# NOTE: When the dimension that we want to add has length greater than 1, the +# values of the data are repeated for that new dimension. -# Example with sample data: -# Check original dimensions and coordinates +# (1) Check original dimensions and coordinates lonlat_temp$exp$dims names(lonlat_temp$exp$coords) -# Add 'variable' dimension +# (2) Add 'variable' dimension exp <- CST_InsertDim(lonlat_temp$exp, posdim = 2, lendim = 2, name = "variable", values = c("tas", "tos")) -# Check new dimensions and coordinates +# (3) Check new dimensions and coordinates exp$dims exp$coords$variable # We see that the values will be repeated along the new dimension: @@ -59,13 +60,21 @@ exp$data[, , 1, 1, 1, 1, 1] #----------------------------------------------------- # In this example, we will merge the dimensions corresponding to the latitude # and the longitude of the data. The new dimension will be named 'grid'. + +# (1) Call the function: new_data <- CST_MergeDims(lonlat_temp$exp, merge_dims = c('lat', 'lon'), rename_dim = 'grid') + +# (2) Check the dimensions of the data: dim(new_data$data) # dataset member sdate ftime grid # 1 15 6 3 1166 + +# (3) Check the names of the coordinates: names(new_data$coords) # [1] "dataset" "member" "sdate" "ftime" "grid" + +# (4) Explore the object by printing it in the terminal: new_data # NOTE: Be aware that when we print the object, we see that its name in # "Coordinates" field appears without the asterisk (*) at its left. This means @@ -75,65 +84,82 @@ attributes(new_data$coords$grid) # $indices # [1] TRUE -# Now, we want to merge time dimensions start date and forecast time: +# (5) Now, we want to merge time dimensions start date and forecast time: new_data <- CST_MergeDims(data = lonlat_temp_st$exp, merge_dims = c('sdate', 'ftime')) # In this case, the Dates dimensions will be merged too. +# (6) Check the dimensions of Dates: dim(new_data$attrs$Dates) # sdate # 18 -# However, when we want to merge temporal and other dimensions nature, -# the Dates dimensions are kept as the original. In this case, the function -# returns a Warning Message, we must pay attention! +# NOTE: When we want to merge temporal and other dimensions nature, +# the Dates dimensions are kept as the original. In this case, the function +# returns a Warning Message, we must pay attention! new_data <- CST_MergeDims(data = lonlat_temp$exp, merge_dims = c('lat', 'ftime'), rename_dim = 'newdim') #----------------------------------------------------- -# Example 4: Split two dimensions with CST_SplitDim +# Example 4: Split two dimensions with SplitDim and CST_SplitDim #----------------------------------------------------- -# In this example, we will start working first with the function SplitDim, +# In this example, we will start working with the function SplitDim, # that it can be used to split dimensions of an array. # NOTE: Take into account that time dimensions will be treated differently than # other dimensions: -# Decadal example: We define an array of consecutive days of different years: +# (1) Decadal example: We define an array of consecutive days of different years: dates <- seq(as.Date("01-01-2000", "%d-%m-%Y", tz = 'UTC'), as.Date("31-12-2005","%d-%m-%Y", tz = 'UTC'), "day") dim(dates) <- c(time = 2192) -# Now, we will split the array in a new 'year' dimension: +# (2) Now, we will split the array in a new 'year' dimension: dates_year <- SplitDim(dates, indices = dates, split_dim = 'time', freq = 'year') # time year # 366 6 -# Now, we can try: freq = 'month' and 'day' + +# (3) Now, we can try: freq = 'month' and 'day' dates_month <- SplitDim(dates, indices = dates, split_dim = 'time', freq = 'month') dates_day <- SplitDim(dates, indices = dates, split_dim = 'time', freq = 'day') -dates <- as.POSIXct(dates * 24 * 3600, origin = '1970-01-01', tz = 'UTC') + +# (4) Finnally, we need to convert them again from numeric to 'POSIXct': +dates_year <- as.POSIXct(dates_year * 24 * 3600, origin = '1970-01-01', tz = 'UTC') +dates_month <- as.POSIXct(dates_month * 24 * 3600, origin = '1970-01-01', tz = 'UTC') +dates_day <- as.POSIXct(dates_day * 24 * 3600, origin = '1970-01-01', tz = 'UTC') + +#----------------------------------------------------- # In the following example, we will use the sample data of the package. We # will use lonlat_prec_st because it is daily data: # NOTE: By Jan 2024, a development is needed regarding updates in other fields # of the 's2dv_cube' + +# (1) Call the function CST_SplitDim with adding 'day' dimension: data_day <- CST_SplitDim(lonlat_prec_st, indices = lonlat_prec_st$attrs$Dates[1,], split_dim = 'ftime', freq = 'day') +# (2) Explore the dimensions of the data array dim(data_day$data) # dataset var member sdate ftime lat lon day # 1 1 6 3 1 4 4 31 + +# (3) Call the function CST_SplitDim with adding 'month' dimension: data_month <- CST_SplitDim(lonlat_prec_st, indices = lonlat_prec_st$attrs$Dates[1,], split_dim = 'ftime', freq = 'month') + dim(data_month$data) # dataset var member sdate ftime lat lon month # 1 1 6 3 31 4 4 1 + +# (4) Call the function CST_SplitDim with adding 'year' dimension: data_year <- CST_SplitDim(lonlat_prec_st, indices = lonlat_prec_st$attrs$Dates[,1], split_dim = 'sdate', freq = 'year') dim(data_year$data) # dataset var member sdate ftime lat lon year # 1 1 6 1 31 4 4 3 + ################################################################################ \ No newline at end of file diff --git a/inst/doc/usecase/ex4_subset.R b/inst/doc/usecase/ex4_subset.R index d86b227e..360ca08f 100644 --- a/inst/doc/usecase/ex4_subset.R +++ b/inst/doc/usecase/ex4_subset.R @@ -3,24 +3,25 @@ # Author: Eva Rifà Rovira # Date: 16/11/2024 #******************************************************************************* -# This example shows how to subset an array. The basic function to subset -# arrays is Subset, from the package ClimProjDiags. Then, in CSTools, there is -# the CST version of the function to subset all other related information -# of the object 's2dv_cube'. -#------------------------------------------------------------------------------- -# Needed packages before a new version is installed +# This example shows how to subset any dimension of an 's2dv_cube'. To do it, +# we will use the function CST_Subset. This function is the 's2dv_cube' method +# version of Subset from the package ClimProjDiags. +# (1) First we will see how Subset works. +# (2) Then, we will use CST_Subset with an 's2dv_cube' + +# Needed packages: library(CSTools) library(ClimProjDiags) -source("https://earth.bsc.es/gitlab/es/ClimProjDiags/-/raw/master/R/Subset.R") -source("https://earth.bsc.es/gitlab/external/cstools/-/raw/master/R/CST_Subset.R") ################################################################################ #----------------------------------------------------- # Example 1: Subset an example array #----------------------------------------------------- -# (1.1) Minimal use case. Spatial coordinates subset. +# This is a minimal use case about spatial coordinates subset. + +# (1) We create the array amd we print it: dat <- array(1:100, dim = c(lat = 10, lon = 10)) -# > dat +dat # [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] # [1,] 1 11 21 31 41 51 61 71 81 91 # [2,] 2 12 22 32 42 52 62 72 82 92 @@ -32,9 +33,11 @@ dat <- array(1:100, dim = c(lat = 10, lon = 10)) # [8,] 8 18 28 38 48 58 68 78 88 98 # [9,] 9 19 29 39 49 59 69 79 89 99 # [10,] 10 20 30 40 50 60 70 80 90 100 + +# (2) We call the function Subset from ClimProjDiags and we see the result: dat_subset <- Subset(x = dat, along = c('lat', 'lon'), indices = list(1:5, 1:7), drop = 'all') -# > dat_subset +dat_subset # [,1] [,2] [,3] [,4] [,5] [,6] [,7] # [1,] 1 11 21 31 41 51 61 # [2,] 2 12 22 32 42 52 62 @@ -45,17 +48,18 @@ dat_subset <- Subset(x = dat, along = c('lat', 'lon'), indices = list(1:5, 1:7), #----------------------------------------------------- # Example 2: Subset an 's2dv_cube' using sample data #----------------------------------------------------- -# (2.1) We don't want to drop any dimension, select only the first member, -# the first and the second start dates, and also subset the longitude and -# keep only the values from [0, 21]: +# In this example we will not drop any dimension, we will select only the first +# member, the first and the second start dates, and also subset the longitude and +# keep only the values from [0, 21]: +# (1) Explore the sample data: dat <- lonlat_temp_st$exp -# > dat$dims +dat$dims # dataset var member sdate ftime lat lon # 1 1 15 6 3 22 53 -# > dat +dat # 's2dv_cube' # Data [ 279.994110107422, 280.337463378906, 279.450866699219, ... ] # Dimensions ( dataset = 1, var = 1, member = 15, sdate = 6, ftime = 3, @@ -90,10 +94,12 @@ dat <- lonlat_temp_st$exp # ( dat1 ) : dataset = dat1, var = tas, sdate = 20001101 ... # ... +# (2) Call the function CST_Subset: dat_subset <- CST_Subset(x = dat, along = c('member', 'sdate', 'lon'), indices = list(1, 1:2, 1:22), drop = 'none') - -# > dat_subset + +# (3) Explore the 's2dv_cube' +dat_subset # 's2dv_cube' # Data [ 279.994110107422, 277.161102294922, 278.825836181641, 276.8271484375, 276.052703857422, 276.950805664062, 280.677215576172, 277.285247802734 ... ] # Dimensions ( dataset = 1, var = 1, member = 1, sdate = 2, ftime = 3, lat = 22, lon = 22 ) @@ -130,4 +136,5 @@ dat_subset <- CST_Subset(x = dat, along = c('member', 'sdate', 'lon'), # load_parameters : # ( dat1 ) : dataset = dat1, var = tas, sdate = 20001101 ... # ... + ################################################################################ \ No newline at end of file -- GitLab