# Hands-on 3: Load data by startR

## Goal
Use startR to load the data used in CSTools [RainFARM vignette](https://earth.bsc.es/gitlab/external/cstools/-/blob/master/vignettes/RainFARM_vignette.Rmd). Learn how to adjust data while loading data.

## 0. Load required packages

```r
# Clean the session
rm(list = ls())

library(startR)
library(CSTools)
```

## 1. Load data from data repository (esarchive/)

**Data description**:  
This sample data set contains a small cutout of gridded seasonal precipitation 
forecast data from the Copernicus Climate Change ECMWF-System 5 forecast system.
Specifically, for the 'prlr' (precipitation) variable, for the first 6 forecast 
ensemble members, daily values, for all 31 days in March following the forecast 
starting dates in November of years 2010 to 2012, for a small 4x4 pixel cutout in 
a region in the North-Western Italian Alps (44N-47N, 6E-9E). The data resolution is 1 degree.

Use the above information to define the variable, start dates, longitude and latitude.

```r
  # Use this one if on workstation or nord3 (have access to /esarchive)
  repos <- "/esarchive/exp/ecmwf/system5c3s/daily_mean/$var$_s0-24h/$var$_$sdate$.nc"
  # Use this one if on Marenostrum4 and log in with PATC2021 account
  repos <- paste0('/gpfs/scratch/nct01/nct01127/d3_R_handson/esarchive/',
                  'exp/ecmwf/system5c3s/daily_mean/',
                  '$var$_s0-24h/$var$_$sdate$.nc')

  var <- 'prlr'
  sdate <- paste0(2010:2012, '1101')
  lon.min <- 6
  lon.max <- 9
  lat.min <- 44
  lat.max <- 47
```

Calculate the forecast time step by package "lubridate". Desired days: the whole March (leap year ignored).
Check the usage of the following functions by type `?<func_name>` or check official website https://lubridate.tidyverse.org/reference/.

```r
  library(lubridate)
  ## Check which year is leap year
  leap_year(2011:2013)
  ## Find the first time step we want
  ftime_s <- as.integer(ymd("2011-03-01", tz = "UTC") - ymd("2010-11-01", tz = "UTC")) + 1
  ftime <- ftime_s:(ftime_s + 30)
  print(ftime)
# [1] 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139
#[20] 140 141 142 143 144 145 146 147 148 149 150 151

```

Use Start() to load the data.

```r
  data <- Start(dat = repos,
                var = var,
                sdate = sdate,
                ensemble = indices(1:6),
                time = ftime,
                latitude = values(list(lat.min, lat.max)),
                latitude_reorder = Sort(),
                longitude = values(list(lon.min, lon.max)),
                longitude_reorder = CircularSort(0, 360),
                synonims = list(latitude = c('lat', 'latitude'),
                                longitude = c('lon', 'longitude')),
                return_vars = list(time = 'sdate',
                                   longitude = NULL, latitude = NULL),
                retrieve = TRUE)
```
   

## 2. Compare with CSTools lonlat_prec

Load CSTools data `lonlat_prec`.

```r
  stored_data <- CSTools::lonlat_prec
```

### Dimensions

```r
  dim(data)
#      dat       var     sdate  ensemble      time  latitude longitude 
#        1         1         3         6        31         4         4 

  dim(stored_data$data)
#dataset  member   sdate   ftime     lat     lon 
#      1       6       3      31       4       4 
```

### Dates

```r
  data_time_attr <- attr(data, 'Variables')$common$time
  dim(data_time_attr)
#sdate  time 
#    3    31 

  dim(stored_data$Dates$start)
#NULL

  str(stored_data$Dates$start)
# POSIXct[1:93], format: "2011-03-01" "2011-03-02" "2011-03-03" "2011-03-04" "2011-03-05" ...

```

### Latitude and Longitude

```r
  data_lat_attr <- attr(data, 'Variables')$common$latitude
  as.vector(data_lat_attr)
#[1] 44 45 46 47

  as.vector(stored_data$lat)
#[1] 47 46 45 44

  data_lon_attr <- attr(data, 'Variables')$common$longitude
  as.vector(data_lon_attr)
#[1] 6 7 8 9

  as.vector(stored_data$lon)
#[1] 6 7 8 9

```

## 3. Adjust Start() call to get more consistent result with lonlat_prec

You should already see that the output of Start() `data` does have the same values
as `lonlat_prec`, but there are some differences. For example, the "member" and "time"
dimensions have different names; the dimension orders are different; latitude vector order is different.

1. We can use parameter `synonims` to change the dimension name freely.
In Start(), change `ensemble` to `member` and `time` to `ftime`, and define their synonims.

2. Sort() in startR is a wrapper function of base function sort(). You can find it in Start()
call above, `latitude_reorder = Sort()`. Type `?sort` in R console to find how to change
latitude to descending order.

3. To switch `member` and `sdate` dimension order, simply switch the order in Start() call.

```r
  data <- Start(dat = repos,
                var = var,
                member = indices(1:6),
                sdate = sdate, 
                ftime = ftime,
                latitude = values(list(lat.min, lat.max)),
                latitude_reorder = Sort(decreasing = TRUE),
                longitude = values(list(lon.min, lon.max)),
                longitude_reorder = CircularSort(0, 360),
                synonims = list(latitude = c('lat', 'latitude'),
                                longitude = c('lon', 'longitude'),
                                member = c('member', 'ensemble'),
                                ftime = c('ftime', 'time')),
                return_vars = list(ftime = 'sdate',
                                   longitude = NULL, latitude = NULL),
                retrieve = TRUE)

  dim(data)
#      dat       var    member     sdate     ftime  latitude longitude 
#        1         1         6         3        31         4         4 

  data_lat_attr <- attr(data, 'Variables')$common$latitude
  as.vector(data_lat_attr)
#[1] 47 46 45 44

```
