---
author: "Nicolau Manubens"
date: "`r Sys.Date()`"
output: html_document
vignette: >
  %\VignetteEngine{R.rsp::md}
  %\VignetteIndexEntry{Example}
  %\usepackage[utf8]{inputenc}
---
Example
=======

Next you can see an example of usage of s2dverification spanning its
four modules ([**Data retrieval**](data_retrieval.md), 
[**Statistics**](statistics.md), [**Verification**](verification.md) 
and [**Visualisation**](visualisation.md)).

The goal of the example is to load and analyse the skill of two sample 
forecasts of the near-surface air temperature over the North Pacific region 
and assess their skill against a reference reanalysis.  
The data used in this example has gone already through a first post-processing
stage in which the monthly means have been computed.

Loading data
------------

First the package is loaded and attached. 

The goal is to load data for two experimental datasets and one observational
dataset to be used as reference. The distribution in the file system and 
details of these datasets are explained in [**Data retrieval**].
 
A list is built with information on the location of each dataset to load.
A path pattern is specified for each dataset, in this case, with the wildcards
'$VAR_NAME$', '$START_DATE$', '$YEAR$', '$MONTH$', '$EXP_NAME$' and 
'$OBS_NAME$'. These wildcards will be automatically replaced by `Load()` with
the corresponding values (the requested variable name, the requested starting
dates, years or months, the dataset names, ...). See the available wildcards in `?Load`.

```r
library(s2dverification)

expA <- list(name = 'experimentA',
             path = file.path('/path/to/experiments/$EXP_NAME$/monthly_mean',
                              '$VAR_NAME$/$VAR_NAME$_$START_DATE$.nc'))
expB <- list(name = 'experimentB',
             path = file.path('/path/to/experiments/$EXP_NAME$/monthly_mean',
                              '$VAR_NAME$/$VAR_NAME$_$START_DATE$.nc'))
obsX <- list(name = 'observationX',
             path = file.path('/path/to/observations/$OBS_NAME$/monthly_mean',
                              '$VAR_NAME$/$VAR_NAME$_$YEAR$$MONTH$.nc'))
```

Then the data is loaded with `Load()` providing the previously built lists
-to specify the desired datasets- and other parameters to select the Earth
surface region of interest, starting dates and forecast time steps to load
data from.  
In this example, the requested format is 2-dimensional: all the loaded data
will be remapped onto the specified common 'grid' via CDO libraries.

```r
sdates <- paste0(1991:2000, '1101')
data <- Load('tas', list(expA, expB), list(obsX),
             sdates = sdates,
             leadtimemin = 2, leadtimemax = 13,
             latmin = -10, latmax = 60,
             lonmin = 100, lonmax = 250, 
             output = 'lonlat', grid = 't106grid',
             method = 'distance-weighted')
## * The load call you issued is:
## *   Load(var = "tas", exp = list(structure(list(name = "experimentA", path =
## *        "/path/to/experiments/$EXP_NAME$/monthly_mean/$VAR_NAME$/$VAR_NAME$_$START_DATE$.nc"),
## *        .Names = c("name", "path")), structure(list(name =
## *        "experimentB", path =
## *        "/path/to/experiments/$EXP_NAME$/monthly_mean/$VAR_NAME$/$VAR_NAME$_$START_DATE$.nc"),
## *        .Names = c("name", "path"))), obs = list(structure(list(name =
## *        "observationX", path =
## *        "/path/to/observations/$OBS_NAME$/monthly_mean/$VAR_NAME$/$VAR_NAME$_$YEAR$$MONTH$.nc"),
## *        .Names = c("name", "path"))), sdates = c("19911101",
## *        "19921101", ..., "20001101"), grid = "t106grid", output = 
## *        "lonlat", storefreq = "monthly", ...)
## * See the full call in '$load_parameters' after Load() finishes.
## * Fetching first experimental files to work out 'var_exp' size...
## * Exploring dimensions... /path/to/experiments/experimentA/monthly_mean/tas/tas_19911101.nc                  
## * Success. Detected dimensions of experimental data: 2, 5, 10, 12, 63, 134
## * Fetching first observational files to work out 'var_obs' size...
## * Exploring dimensions... /path/to/observations/observationX/monthly_mean/tas/tas_199112.nc                  
## * Success. Detected dimensions of observational data: 1, 1, 10, 12, 63, 134
## * Will now proceed to read and process 140 data files:
## *   The list of files is long. You can check it after Load() finishes in the output '$source_files'.
## * Total size of requested data:  89147520 bytes.
## *   - Experimental data:  ( 2 x 5 x 10 x 12 x 63 x 134 ) x 8 bytes = 81043200 bytes.
## *   - Observational data: ( 1 x 1 x 10 x 12 x 63 x 134 ) x 8 bytes = 8104320 bytes.
## * If size of requested data is close to or above the free shared RAM memory, R will crash.
## starting worker pid=3674 on localhost:11060 at 15:11:26.226
## starting worker pid=3690 on localhost:11060 at 15:11:26.419
## starting worker pid=3706 on localhost:11060 at 15:11:26.614
## starting worker pid=3722 on localhost:11060 at 15:11:26.805
## starting worker pid=3739 on localhost:11060 at 15:11:26.997
## starting worker pid=3755 on localhost:11060 at 15:11:27.190
## starting worker pid=3771 on localhost:11060 at 15:11:27.380
## starting worker pid=3787 on localhost:11060 at 15:11:27.572
## * Loading... This may take several minutes...
## * Progress: 0% + 10% + 10% + 10% + 10% + 10% + 10% + 10% + 10% + 10% + 10%
```

The output consists of two arrays of data (experimental and observational
data) with labelled dimensions, a list of loaded files, a list of not found
files and a call stamp to exactly reproduce as needed, among others.  
See [**Data
retrieval**](data_retrieval.md)
for a full explanation of the capabilities and outputs of `Load()`.

Keep in mind that the two arrays obtained from `Load()` will have the 
following dimensions in the following order:
```r
  c(n. of datasets, n. of members, n. of starting dates, n. of lead-times, n. of latitudes, n. of longitudes)
```

Checking and summarizing data
-----------------------------

<a name="snippet1"></a>
Next, the data of the first member, starting date and leadtime of each dataset
can be visualised on a cylindrical equidistant projection with `PlotEquiMap()` 
to check that the loaded data is as expected.
```r
PlotEquiMap(data$mod[1, 1, 1, 1, , ], data$lon, data$lat)
PlotEquiMap(data$mod[2, 1, 1, 1, , ], data$lon, data$lat)
PlotEquiMap(data$obs[1, 1, 1, 1, , ], data$lon, data$lat)
```
<img src="snip1_equi_map_raw_all.png" />

See the full code used to obtain this figure in 
[**Snippet 1**](snippets.md#snippet1).

The values for some starting dates at a single grid point of an experimental
dataset can be plotted together with the observations with the function 
`PlotAno()`:
```r
mod <- data$mod[, , , , 30, 60, drop = FALSE]
dim(mod) <- dim(mod)[1:4]
obs <- data$obs[, , , , 30, 60, drop = FALSE]
dim(obs) <- dim(obs)[1:4]
PlotAno(mod, obs, gsub('1101', '1201', sdates),
        toptitle = paste0("Experiment ", c("A", "B"), 
                          ": raw 'tas' at 166.5ºE, 27.47ºN."), 
        ytitle = "K",
        fileout = c('ex_raw_expA_obsX.eps', 'ex_raw_expB_obsX.eps'))
```

<img src="ex_raw_expA_obsX.png" width="600" />
<img src="ex_raw_expB_obsX.png" width="600" />

Each coloured region represents data corresponding to a single starting date.  
The bold line represents the mean value and the thin lines represent the values
of each ensemble member. The upper and lower limits of each region are 
defined, respectively, by the maximum and minimum values among the members.  
The black line represents the reference reanalysis values.

We can see in the plot how the 'tas' reaches a minimum in February/March.

We can work out the exact coordinates of the selected grid cell as follows:
```r
data$lon[60]
## 166.5
data$lat[30]
## 27.47649
```

At this point, additional numerical checks could be carried out to ensure the
data retrieval stage has been sucessful.

Bias correction
---------------

The next common step is to perform statistics on the raw data such as
computing climatologies and subtracting them to the raw data to obtain 
anomalies that account for the inherent drift of the forecasts.  
This is straightforward with the functions `Clim()` and `Ano()` (or 
`Ano_CrossValid()`, see [**Statistics**]). Before, however, it is usual to 
average the data over the region of interest instead of focusing on a single 
grid cell. This can be done by directly requesting the data to `Load()` with 
the 'areave' output type, which will apply area weights and calculate the 
total average:
```r
data <- Load('tas', list(expA, expB), list(obsX),
             sdates = sdates,
             leadtimemin = 2, leadtimemax = 13,
             latmin = -10, latmax = 60,
             lonmin = 100, lonmax = 250, 
             output = 'areave', grid = 't106grid',
             method = 'distance-weighted')
```

Then we calculate and plot the per-pair climatologies with `Clim()` and 
`PlotClim()`:
```r
clim <- Clim(data$mod, data$obs)
PlotClim(clim$clim_exp, clim$clim_obs, monini = 12,
         toptitle = "Per-pair climatologies of 'tas' over North Pacific region.",
         listexp = c('Experiment A', 'Experiment B'),
         listobs = c('Observation X'),
         ytitle = "K", fileout = 'ex_clim_expA_expB_obsX.eps')
```

<img src="ex_clim_expA_expB_obsX.png" width="600" />

Each line in this plot represents the climatology of each member of the 
corresponding dataset. A single climatology of the ensemble mean could be 
obtained providing the parameter memb = FALSE to `Clim()`, if distinction 
of members is irrelevant.

We can obtain the anomalies with `Ano()`, which simply subtracts the 
climatologies to the corresponding starting dates and members of the raw data.
These can be plotted with `PlotAno()`:
```r
ano_mod <- Ano(data$mod, clim$clim_exp)
ano_obs <- Ano(data$obs, clim$clim_obs)
PlotAno(ano_mod, ano_obs, gsub('1101', '1201', sdates), 
        toptitle = paste0("Experiment ", c("A", "B"), 
                          ": drift-corrected 'tas' anomalies over North Pacific region."),
        ytitle = 'K', linezero = TRUE,
        fileout = c('ex_ano_expA_obsX.eps', 'ex_ano_expB_obsX.eps'))
```

<img src="ex_ano_expA_obsX.png" width="600" />
<img src="ex_ano_expB_obsX.png" width="600" />

To fulfill the bias correction we would need to add the observed climatologies 
to these anomalies. The working units of the package, however, are the 
anomalies so this step is not needed.

Other statistics such as trends, detrended anomalies, frequency filtering, 
EOF/PCA and empirical model generation are implemented in the package.  
Check [**Statistics**](statistics.md) for a thorough explanation.
Also see [**Visualisation**](visualisation.md) for details on visualisation 
tools.

Assessing the quality
---------------------

As a measure of the skill of the two forecasts we will compute their 
correlation against the reanalysis.

First we compute the ensemble mean and then calculate the Pearson correlation
with `Corr()` and plot the results with `PlotVsLTime()`, which allows to plot
any other computed variable with confidence intervals and signification values.
```r
corr <- Corr(Mean1Dim(ano_mod, 2), Mean1Dim(ano_obs, 2))
PlotVsLTime(corr, toptitle = "Correlations with Observation X over North Pacific region.",
            ytitle = 'corr', monini = 12, 
            listexp = c('Exp A', 'Exp B'),
            fileout = 'ex_corr_expA_expB_obsX.eps')
```

<img src="ex_corr_expA_expB_obsX.png" width="600" />

See [**Verification**](verification.md) for a detailed explanation of the 
available deterministic and probabilistic scores or 
[**Plotting time series**](visualisation.md#time_series) for quick reference of
how each score can be plotted.
