---
title: "Data Loading"
author: "An-Chi Ho"
date: "`r Sys.Date()`"
output: html_document
vignette: >
  %\VignetteEngine{R.rsp::md}
  %\VignetteIndexEntry{Data Loading}
  %\usepackage[utf8]{inputenc}
---

# Data Loading

To exihibit the usage of the plotting functions, we need to have data loaded first.
Here, we use `startR` package to load the experimental and observational data from our data archive (esarchive).
If you don't have access to esarchive, you can contact the package maintainer (an.ho@bsc.es; eva.rifarovira@bsc.es)
to obtain the sample data. 

We are going to load three datasets: the experiment data are SEAS5 and Meteo-France System 7 from ECMWF, and the observation data ERA5 from ECMWF.

In vignettes [Time Series](time_series.md) and [Map: Equidistant Projection](map_proj_equi.md), we use Europe region (20W-40E, 20N-80N); 
in [Map: Stereographic Projection](map_proj_stereo.md), we use global region; 
and in [Map: Various Projection](map_proj_robinson.md), we use both. 
Adjust the variable `region` in the script to load different region.

The variable to be analyzed is the monthly mean near-surface temperature (tas). 
The forecast years are 1997 to 2016 for european region and 2011 to 2016 for global region. 
The initial month is November. All the ensemble members and the first 6 forecast time steps are loaded.

## Load monthly mean data

```r
library(startR)

region <- "euro"
if (region == "euro") {
  lat_min <- 20
  lat_max <- 80
  lon_min <- -20
  lon_max <- 40
  sdates <- paste0(1997:2016, '1101')

} else if (region == "global") {
  lat_min <- -90
  lat_max <- 90
  lon_min <- -180
  lon_max <- 179.99
  sdates <- paste0(2011:2016, '1101')

}

repo_exp1 <- paste0('/esarchive/exp/ecmwf/system5c3s/monthly_mean/',
                    '$var$_f6h/$var$_$sdate$.nc')
repo_exp2 <- paste0('/esarchive/exp/meteofrance/system7c3s/monthly_mean/',
                    '$var$_f6h/$var$_$sdate$.nc')

exp <- Start(dat = list(list(name = "system5c3s", path = repo_exp1),
                        list(name = "meteofrance", path = repo_exp2)),
             var = 'tas',
             sdate = sdates,
             time = 1:6,
             ensemble = 'all',
             lat = values(list(lat_min, lat_max)),
             lat_reorder = Sort(),
             lon = values(list(lon_min, lon_max)),
             lon_reorder = CircularSort(-180, 180),
             transform = CDORemapper,
             transform_params = list(grid = 'r360x181', method = 'bilinear'),
             transform_vars = c('lat', 'lon'),
             return_vars = list(lat = 'dat', lon = 'dat',
                                time = 'sdate'),
             retrieve = TRUE)

# Use exp time metadata to load obs
sdate_obs <- format(attributes(exp)$Variables$common$time, "%Y%m")
dim(sdate_obs) <- dim(attributes(exp)$Variables$common$time)

repo_obs <- '/esarchive/recon/ecmwf/era5/monthly_mean/$var$_f1h-r1440x721cds/$var$_$sdate$.nc'

obs <- Start(dat = repo_obs,
             var = 'tas',
             sdate = sdate_obs,
             split_multiselected_dims = TRUE,
             lat = values(list(lat_min, lat_max)),
             lat_reorder = Sort(),
             lon = values(list(lon_min, lon_max)),
             lon_reorder = CircularSort(-180, 180),
             transform = CDORemapper,
             transform_params = list(grid = 'r360x181', method = 'bilinear'),
             transform_vars = c('lat', 'lon'),
             synonims = list(lat = c('lat', 'latitude'),
                             lon = c('lon', 'longitude')),
             return_vars = list(time = 'sdate',
                                lon = NULL, lat = NULL),
             retrieve = TRUE)
```

## Calculate spatial mean

To plot the time series type, we calculate the spatial mean by `ClimProjDiags::WeightedMean`,
and remove the redundant "var" dimension.

```r
lat <- attributes(exp)$Variables$system5c3s$lat
lon <- attributes(exp)$Variables$system5c3s$lon

exp_sm <- ClimProjDiags::WeightedMean(exp, lat = lat, lon = lon, latdim = "lat", londim = "lon", na.rm = T)
dim(exp_sm) <- dim(exp_sm)[c("dat", "sdate", "time", "ensemble")]

obs_sm <- ClimProjDiags::WeightedMean(obs, lat = lat, lon = lon, latdim = "lat", londim = "lon", na.rm = T)
dim(obs_sm) <- dim(obs_sm)[c("dat", "sdate", "time")]
```


Now, we have both the original data and the spatial mean data. We can move to other vignettes to see the usage of the plotting functions:
- [**Time Series Plotting**](time_series.md): To plot different types of time series data
- [**Map: Equidistant Projection**](map_proj_equi.md
): Introduce the functions that plot maps with equidistant projection 
- [**Map: Stereographic Projection**](map_proj_stereo.md
): Introduce the functions that plot maps with stereographic projection 
- [**Map: Various Projections**](map_proj_robinson.md): To plot maps with various projections 
