# Hands-on 2: Data assesment

**Goal**

Use CSTools and s2dv to calibrate, compute anomalies and skill of a climate dataset over a period.

**Load packages**
```r
library(CSTools)
library(s2dv)
```

## 1. Load the data

In this section we will use the function **Start** to load the data. Then, we will transfrom the output `startR_array` to an `s2dv_cube` object in order that the data is easy to use within CSTools functions.

The `s2dv_cube` object is a structured list that contains the information needed to work with multidimensional arrays of data. Coordinates, dimensions, and metadata are neatly combined to allow for a more intuitive, concise, and less error-prone experience.

**Note:** If you have already loaded the data with Start, go directly to section 1b).

### 1a) Load the data

The following section is taken from [PATC 2023 startR tutorial](https://earth.bsc.es/gitlab/es/startR/-/blob/doc-bsc_training_2023/inst/doc/tutorial/PATC2023/handson_1-data-loading.md?ref_type=heads). The experiment data are Meteo-France System 7 from ECMWF, and the observation ones are ERA5 from ECMWF. We're going to analyze the near-surface temperature (short name: tas) for seasonal forecast. We will focus on the Europe region (roughly 20W-40E, 20N-80N). The hindcast years are 1993 to 2016, and the forecast year is 2020. The initial month is November. 

```r
# Use this one if on workstation or nord3 (have access to /esarchive)
path_exp <- "/esarchive/exp/meteofrance/system7c3s/monthly_mean/$var$_f6h/$var$_$syear$.nc"
#----------------------------------------------------------------------
# Run these two lines if you're on Marenostrum4 and log in with training account
prefix <- '/gpfs/scratch/bsc32/bsc32734/bsc_training_2023/R_handson/'
path_exp <- paste0(prefix, path_exp)
#----------------------------------------------------------------------

sdate_hcst <- paste0(1993:2016, '1101')

hcst <- CST_Start(dat = path_exp,
                  var = 'tas',
                  syear = sdate_hcst,
                  ensemble = 'all',
                  time = 1:2,
                  latitude = startR::values(list(20, 80)),
                  latitude_reorder = startR::Sort(),
                  longitude = startR::values(list(-20, 40)),
                  longitude_reorder = startR::CircularSort(-180, 180),
                  transform = startR::CDORemapper,
                  transform_params = list(grid = 'r360x181', method = 'bilinear'),
                  transform_vars = c('latitude', 'longitude'),
                  synonims = list(latitude = c('lat', 'latitude'),
                                  longitude = c('lon', 'longitude')),
                  return_vars = list(time = 'syear',
                                     longitude = NULL, latitude = NULL),
                  retrieve = TRUE)

```

```r
sdate_fcst <- '20201101'

fcst <- CST_Start(dat = path_exp,
                  var = 'tas',
                  syear = sdate_fcst,
                  ensemble = 'all',
                  time = 1:2, 
                  latitude = startR::values(list(20, 80)),
                  latitude_reorder = startR::Sort(),
                  longitude = startR::values(list(-20, 40)),
                  longitude_reorder = startR::CircularSort(-180, 180),
                  transform = startR::CDORemapper,
                  transform_params = list(grid = 'r360x181', method = 'bilinear'),
                  transform_vars = c('latitude', 'longitude'),
                  synonims = list(syear = c('syear', 'sdate'),
                                  latitude = c('lat', 'latitude'),
                                  longitude = c('lon', 'longitude')),
                  return_vars = list(time = 'syear',
                                    longitude = NULL, latitude = NULL),
                  retrieve = TRUE)

```

```r
# Adjust the day to the correct month
hcst$attrs$Dates <- hcst$attrs$Dates - lubridate::days(1)

date_string <- format(hcst$attrs$Dates, '%Y%m')
sdate_obs <- array(date_string, dim = c(syear = 24, time = 2))
```

```r
path_obs <- '/esarchive/recon/ecmwf/era5/monthly_mean/$var$_f1h-r1440x721cds/$var$_$syear$.nc'
#----------------------------------------------------------------------
# Run these two lines if you're on Marenostrum4 and log in with training account
prefix <- '/gpfs/scratch/bsc32/bsc32734/bsc_training_2023/R_handson/'
path_obs <- paste0(prefix, path_obs)
#----------------------------------------------------------------------

obs <- CST_Start(dat = path_obs,
                 var = 'tas',
                 syear = sdate_obs,
                 split_multiselected_dims = TRUE,
                 latitude = startR::values(list(20, 80)),
                 latitude_reorder = startR::Sort(),
                 longitude = startR::values(list(-20, 40)),
                 longitude_reorder = startR::CircularSort(-180, 180),
                 transform = startR::CDORemapper,
                 transform_params = list(grid = 'r360x181', method = 'bilinear'),
                 transform_vars = c('latitude', 'longitude'),
                 synonims = list(syear = c('syear', 'sdate'),
                                 latitude = c('lat', 'latitude'),
                                 longitude = c('lon', 'longitude')),
                 return_vars = list(time = 'syear',
                                    longitude = NULL, latitude = NULL),
                 retrieve = TRUE)

```

### 1b) Create `s2dv_cube`:

Now we convert the data (`startR_array`) into an 's2dv_cube' object with the function **as.s2dv_cube**:

```r
> str(hcst)
 'startR_array' num [1, 1, 1:24, 1:25, 1:2, 1:60, 1:60] 295 296 297 297 297 ...
 - attr(*, "Variables")=List of 2
  ..$ common:List of 4
  .. ..$ time     : POSIXct[1:48], format: "1993-12-01" "1994-12-01" ...
  .. ..$ longitude: num [1:60(1d)] -19.5 -18.5 -17.5 -16.5 -15.5 -14.5 -13.5 -12.5 -11.5 -10.5 ...
  .. .. ..- attr(*, "variables")=List of 1
  .. .. .. ..$ longitude:List of 7
  .. .. .. .. ..$ ndims        : num 1
  .. .. .. .. ..$ size         : int 360
  .. .. .. .. ..$ units        : chr "degrees_east"
  .. .. .. .. ..$ dim          :List of 1
  .. .. .. .. .. ..$ :List of 10

```
Now, we convert the startR object into 's2dv_cube' object.
```r
hcst <- as.s2dv_cube(hcst)
fcst <- as.s2dv_cube(fcst)
obs <- as.s2dv_cube(obs)
```

```r
> str(hcst)
List of 4
 $ data  : num [1, 1, 1:24, 1:25, 1:2, 1:60, 1:60] 295 296 297 297 297 ...
 $ dims  : Named int [1:7] 1 1 24 25 2 60 60
  ..- attr(*, "names")= chr [1:7] "dat" "var" "syear" "ensemble" ...
 $ coords:List of 7
  ..$ dat      : chr "dat1"
  .. ..- attr(*, "indices")= logi FALSE
  ..$ var      : chr "tas"
  .. ..- attr(*, "values")= logi TRUE
  .. ..- attr(*, "indices")= logi FALSE
  ..$ syear    : chr [1:24] "19931101" "19941101" "19951101" "19961101" ...
  .. ..- attr(*, "values")= logi TRUE
  .. ..- attr(*, "indices")= logi FALSE
  ..$ ensemble : int [1:25] 1 2 3 4 5 6 7 8 9 10 ...
  .. ..- attr(*, "indices")= logi TRUE
  ..$ time     : int [1:2] 1 2
  .. ..- attr(*, "indices")= logi TRUE
  ..$ latitude : num [1:60] 20.5 21.5 22.5 23.5 24.5 25.5 26.5 27.5 28.5 29.5 ...
  .. ..- attr(*, "variables")=List of 1
  .. .. ..$ latitude:List of 7
  .. .. .. ..$ ndims        : num 1
  [...]
```
**Questions 1:**   

Find `s2dv_cube` information of `hcst` object. Here there are some useful functions:
```r
str()
typeof()
class()
dim()
names()
summary()
```
```r
`s2dv_cube` overview
$ data: [data array]
$ dims: [dimensions vector]
$ coords: [List of coordinates vectors]
  [...]
$ attrs: [List of the attributes]
  $ Dates 
  $ Variable:
    $ varName
    $ metadata 
  [...]
```

1. Can you tell what type of object is an **s2dv_cube** in R structures?  
```r
class(hcst)
# "s2dv_cube"
typeof(___)

```
1. Can you tell what type of object is the element data in common language? Use the function `dim()` and `typeof()` to check `hcst$data`
```r
dim(____)

typeof(hcst$data) 

```
2. The Dates of an `s2dv_cube` can be found in element: `hcst$attrs$Dates`. 
Could you tell what are the time dimensions of the object?  
```r

```
3. The coordinates in the  `s2dv_cube` are stored in element `hcst$coords`. What are the coordinates names in the object `hcst`? Use the function `names()` to check:
```r
names(____)

```
4.  In which region we have loaded the data?  
```r

```
5. What is the start date dimension name? What is the ensemble member dimension name?
```r
hcst$dims
# syear
# ensemble
```

6. How many ensemble members have the `hcst`, `fcst` and `obs` datasets? 
```r

```
7. The metadata of the `s2dv_cube` is stored in `hcst$attrs$Variable` What is the full variable name of the loaded data? Find out the information in `hcst$attrs$Variable$metadata` with the function `str()`:
```r

```
8. What season is the data loaded from? You can use the function `month()`
```r

```
9. What are the units of the data?
```r

```

**Exercise 1**:   
1. Find the mean, the maximum and the minimum of the `fcst`, `hcst` and obs data and compare it:

```r

```

## 2. Calibrate the data
Now, we are going to use the function CST_Calibration from CSTools to calibrate the hindcast. The parameter `cal.method` alllows us to chose the calibration method. In this exercise we are going to chose the `evmos`.  

The "evmos" method applies a variance inflation technique to ensure the correction of
the bias and the correspondence of variance between forecast and observation (Van Schaeybroeck and Vannitsem, 2011).
```r
hcst_cal <- CST_Calibration(exp = hcst, obs = obs,
                            cal.method = "evmos",
                            eval.method = "leave-one-out",
                            multi.model = FALSE,
                            na.fill = TRUE,
                            na.rm = TRUE,
                            apply_to = NULL,
                            alpha = NULL,
                            memb_dim = "ensemble",
                            sdate_dim = "syear",
                            ncores = 10)
```

Now we are going to calibrate also the forecast:
```r

fcst_cal <- CST_Calibration(exp = hcst, obs = obs, exp_cor = fcst,
                            cal.method = "evmos",
                            eval.method = "leave-one-out",
                            multi.model = FALSE,
                            na.fill = TRUE,
                            na.rm = TRUE,
                            apply_to = NULL,
                            alpha = NULL,
                            memb_dim = "ensemble",
                            sdate_dim = "syear",
                            ncores = 10)
```
**Exercise 2**: 
1. Test the hcst with a different calibration methods available and compare the results. You can use the function `summary()` to see the differences.
```r
hcst_cal_bias <- CST_Calibration(____)
```

## 3. Compute Anomalies

Now, we will compute the hindcast anomalies with the calibrated `hcst`. We are going to use the function CST_Anomaly from CSTools.

This function computes the anomalies relative to a climatology computed along the selected dimension (usually starting dates or forecast time) allowing the application or not of crossvalidated climatologies. The computation is carried out independently for experimental and observational data products.

```r
hcst_anom <- CST_Anomaly(exp = hcst_cal, obs = obs,
                         cross = TRUE,
                         memb = TRUE,
                         memb_dim = 'ensemble',
                         dim_anom = 'syear',
                         dat_dim = c('dat', 'ensemble'),
                         ftime_dim = 'time',
                         ncores = 10)

```
**Exercise 3**: 
1. Calculate the hcst anomalies with the raw hindcast but change the number of ncores.
```r
t1 <- Sys.time()
hcst_anom_raw <- CST_Anomaly(____)
t2 <- Sys.time()
t2-t1
```


## 4. Compute skill: RPSS
In order to trust the results we need to evaluate the skill of the system. 

Compute Ranked Probability Skill Score for anomalies: 
```r
skill <- RPSS(exp = hcst_anom$exp$data, obs = hcst_anom$obs$data,
              time_dim = 'syear',
              memb_dim = 'ensemble',
              Fair = FALSE,
              cross.val = TRUE,
              ncores = 10)
```
**Exercise 4**: 
1. Compare the result with the raw results.

```r
skill_raw <- RPSS(exp = hcst_anom_raw$exp$data, obs = hcst_anom_raw$obs$data,
                  time_dim = 'syear',
                  memb_dim = 'ensemble',
                  Fair = FALSE,
                  cross.val = TRUE,
                  ncores = 10)
> summary(skill$rpss)
    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
-0.54005 -0.11225 -0.03170 -0.03722  0.04480  0.44376 
> summary(skill$sign)
   Mode   FALSE    TRUE 
logical    6798     402 
> summary(skill_raw$rpss)
    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
-0.59113 -0.11045 -0.03069 -0.03585  0.04637  0.44305 
> summary(skill_raw$sign)
   Mode   FALSE    TRUE 
logical    7055     387 
```

## 5. Additional Exercise: Visualization

We can use the function from s2dv PlotEquiMap to visualize the data. 

**Example**

With the following code, we will plot a map to compare the hindcast raw and calibrated data. Can we appreciate the differences? 

We are going to plot the last year of the hindcast period (2016) for the last timestep (December). Also, we are going to use the last ensemble member (arbirtrary choice).
```r
dim(hcst$data)
    #   dat       var     syear  ensemble      time  latitude longitude 
    #     1         1        24        25         2        60        60 
```
```r
# Visualization: compare raw and calibrated data
PlotEquiMap(hcst$data[,,24,1,1,,], lat = lat, lon = lon, 
            filled.continents = FALSE, fileout = "/esarchive/scratch/erifarov/rpackages/PATC2023/figs_res/raw_hcst_24.png")

PlotEquiMap(hcst_cal$data[,,24,1,1,,], lat = lat, lon = lon, 
            filled.continents = FALSE, fileout = "/esarchive/scratch/erifarov/rpackages/PATC2023/figs_res/cal_hcst_24.png")

```
