Visualisation
=============

s2dverification contains a set of functions to plot data at every stage of the 
verification process, most based directly on R graphics plotting tools.
These functions are essential to:
 - Quickly inspect the results of a newly produced experiment, i.e. to check 
the physical consistency of the results.
 - Assess the added value of a new prediction system, i.e. to compare new 
results with a reference (observation, reconstruction or other experiment).
 - Assess visually the significance of results, i.e. to display in a 
user-friendly way confidence intervals and other statistics.

The visualisation functions, most with a name following the pattern 
`Plot*()`, can be grouped as a function of the kind of plot they provide: 
 - [**Plotting time series**](#time_series): `PlotClim()`, `PlotAno()`, 
`PlotVsLtime()`, `Plot2VarsVsLTime()` and `PlotACC()`.
 - [**Plotting 2-dimensional fields**](#2d_fields): `PlotEquiMap()`, 
`PlotStereoMap()`, `AnimVsLTime()`, `PlotSection()`.

To master these functions it is convenient to have in mind the common array 
dimension structure used throughout in s2dverification and how it evolves as 
the data objects go through the statistics and verification stages. For that 
you can review the introduction in [**Data retrieval**](data_retrieval.md) and 
the sections [**Statistics**](statistics.md) and 
[**Verification**](verification.md).

Next an explanation of which situations they fit the best, details of the 
options they provide and short examples of usage. The data used hereunder will
be the same as in [**Statistics**](statistics.md):
```
library(s2dverification)
expA <- list(name = 'experimentA',
             path = file.path('/path/to/experiments/$EXP_NAME$/monthly_mean',
                              '$VAR_NAME$/$VAR_NAME$_$START_DATE$.nc'))
expB <- list(name = 'experimentB',
             path = file.path('/path/to/experiments/$EXP_NAME$/monthly_mean',
                              '$VAR_NAME$/$VAR_NAME$_$START_DATE$.nc'))
obsX <- list(name = 'observationX',
             path = file.path('/path/to/observations/$OBS_NAME$/monthly_mean',
                              '$VAR_NAME$/$VAR_NAME$_$YEAR$$MONTH$.nc'))
sdates <- paste0(1991:2000, '1101')
data <- Load('tas', list(expA, expB), list(obsX),
             sdates = sdates,
             leadtimemin = 2, leadtimemax = 13,
             latmin = -10, latmax = 60,
             lonmin = 100, lonmax = 250,
             output = 'areave', grid = 't106grid',
             method = 'distance-weighted')
ano <- Ano_CrossValid(data$mod, data$obs, memb = FALSE)
```

Plotting time series<a name="time_series"></a>
--------------------

All the functions devoted to plotting time series have some common traits and
parameters:
 - Aimed to plot monthly, seasonal or yearly time series, adjustable via the 
parameter `freq`.
 - All of them generate plots and save them to PostScript files the path and 
file name of which you have to provide to the `fileout` parameter. Can plot in
a presentation oriented style or in a paper oriented style, adjustable with the
parameter `biglab`.
 - Can plot a legend automatically, adjustable via the parameters `leg`, 
`listexp` and `listobs` in `PlotVsLTime()`, `Plot2VarVsLtime()` and 
`PlotClim()` or via the parameter `legends` in `PlotAno()` and `PlotACC()`.
 - Accept any additional parameters via the parameter `...` to be sent to to 
the underlying R graphics functions for a fine tuning.

### Plotting climatologies

`PlotClim()` takes as inputs an experimental and an observational array of 2 
to 3 dimensions each:
```r
  c(n. of datasets, n. of members, n. of lead-times)
```
the second one being optional. This means that the latitude and longitude 
dimensions must have been removed either because working on area-averages or
because working on a single grid point and that the starting dates dimension
must have also been removed either because willing to plot data of a single
starting date or because working on averages across starting dates, i.e. 
climatologies (see [**Statistics**](statistics.md)).

A curve is plotted for each member of each experiment or observation. The 
members of a same experiment or observation are equally coloured whereas the 
different experiments and observations are plotted with different colours.  
`PlotClim()` assumes by default that the provided climatologies start in 
January. Otherwise the initial month can be specified with the parameter 
`monini`:

```r
clim <- Clim(data$mod, data$obs)
PlotClim(clim$clim_exp, clim$clim_obs, monini = 12,
         toptitle = "Per-pair 'tas' climatologies over the Atlantic region.",
         ytitle = "K",
         listexp = c('Experiment A', 'Experiment B'),
         listobs = c('Observation X'),
         fileout = "vis_clim_expA_expB_obsX.eps")
```
<img src="vis_clim_expA_expB_obsX.png" width="600" />

### Plotting multi-member raw data or anomalies

`PlotAno()` takes also two data arrays as inputs with 4 dimensions each:
```r
  c(n. of datasets, n. of members, n. of starting dates, n. of lead-times)
```
all being mandatory. It means that if willing to plot single member data or
data from a single starting date, the corresponding dimensions still have to
be present in the provided arrays; `InsertDim()` helps in that.

The data in the provided arrays can be raw data, anomalies or any other kind
of data, as long as it is arranged with the required dimensions.  
For each run of a model (i.e. for each group of members of a same starting 
date from one experiment) a thick curve with the ensemble mean is plotted 
together with a finer curve for each ensemble member, unless any of these are 
disabled with `ensmean` or `memb`, respectively. The area delimited by the 
minimum and maximum ensemble values at each forecast time is filled in with a 
different colour for each starting date, unless disabled with `fill`.  
A fine black curve is plotted with the observational data alongside each 
starting date.  
A separate new plot is created for each experimental dataset provided in the
experimental data array.

The starting dates the provided data corresponds to must be provided with the
parameter `sdates` for them to be plotted accordingly along the x axis. 

`linezero` draws a line at y = 0 to help perceiving whether the anomaly is 
positive or negative, `vlines` draws vertical lines at any specified abscissae
and `points` will draw all the curves in pointed style.

```r
sdates <- paste0(1991:2000, '1101')
selected_sdates <- gsub('1101', '1201', sdates)
PlotAno(data$mod, data$obs, selected_sdates,
        toptitle = paste0(c("Experiment A", "Experiment B"),
                          ": Raw 'tas' over Atlantic region."),
        ytitle = c("K", "K"),
        fileout = paste0("vis_raw_exp", c("A", "B"), "_obsX.eps"))
```
<img src="vis_raw_expA_obsX.png" width="600" />
<img src="vis_raw_expB_obsX.png" width="600" />

### Plotting statistics and scores

The functions `PlotVsLTime()`, `Plot2VarsVsLTime()` and `PlotACC()` serve to
plot time series of variables obtained with the 
[**Statistics**](statistics.md) or [**Verification**](verification) modules 
that yield forecast time series of indices or scores that usually come 
together with confidence intervals and significance levels.

`PlotVsLTime()` takes as main input an array of data with the following 
dimensions:
```r
  c(n. of experiments, n. of observational datasets, 3 or 4, n. of lead-times)
```
the second one being optional and the third one corresponding to the lower 
limit of a confidence interval, the measurement or index or score, the upper
limit of a confidence interval and, optionally, a significance level. This
dimension format requires that the index or score to be plotted is computed
along the actual time (starting dates) and over the average of the ensemble 
members, either at a single grid point or from an area average.

A curve is drawn for each pair of (experiment, observation) with a different 
colour and line style. The scores or indices that correspond to a same 
experiment are coloured equally but a different line style is used for each 
observation that experiment has been compared to when computing the index or 
score. The confidence intervals are plotted with a finer line of the same
colour and line style, unless disabled with `show_conf`. To plot the 
signification level instead the parameter `siglev` can be set to `TRUE`.

As in `PlotClim()`, each curve is defined along the forecast times and is
assumed to start at January. Otherwise it can be adjusted via `monini`. 
Additionally the number of ticks along the x axis can be adjusted with 
`nticks` and any number of horizontal lines can be drawn by specifying the 
target ordinates to `hlines`. 

`PlotVsLTime()` can be useful in the following situations:
 - To plot `Trend()` or `Regression()` fitted coefficients of ensemble averages 
of data at a single grid cell or averaged over a region:

```r
trend_exp <- Trend(Mean1Dim(ano$ano_exp, 2))
trend_obs <- Trend(Mean1Dim(ano$ano_obs, 2))
PlotVsLTime(trend_exp$trend,
            toptitle = "Slopes of trends of 'tas' over Atlantic region",
            ytitle = "K/year",
            monini = 12, freq = 1,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_trend_expA_expB.eps')
```
```r
ano_expA_Y <- Regression(Mean1Dim(plyr::take(ano$ano_exp, 1, 1), 2),
                         Mean1Dim(plyr::take(ano$ano_exp, 1, 2), 2))
PlotVsLTime(ano_expA_Y$regression,
            toptitle = "Slopes of regression of exp. A with exp. B, over Atlantic region",
            ytitle = "K/year",
            monini = 12, freq = 1, leg = FALSE,
            fileout = 'vis_regression_expA_expB.eps')
```
<img src="vis_trend_expA_expB.png" width="600" />
<img src="vis_regression_expA_expB.png" width="600" />
 - To plot the `Spread()` across ensemble members and starting dates of area 
averaged data (interquartile range, maximum minus minimum, standard deviation
or median absolute deviation):

```r
spread <- Spread(ano$ano_exp, posdim = c(2, 3))
PlotVsLTime(spread$iqr,
            toptitle = "IQR across members and starting dates over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_iqr_expA_expB.eps')
PlotVsLTime(spread$maxmin,
            toptitle = "Max. - min. across members and starting dates over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_maxmin_expA_expB.eps')
PlotVsLTime(spread$sd,
            toptitle = "S. deviation across members and starting dates over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_sd_expA_expB.eps')
PlotVsLTime(spread$iqr,
            toptitle = "MAD across members and starting dates over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_mad_expA_expB.eps')
```
<img src="vis_iqr_expA_expB.png" width="600" />
<img src="vis_maxmin_expA_expB.png" width="600" />
<img src="vis_sd_expA_expB.png" width="600" />
<img src="vis_mad_expA_expB.png" width="600" />
 - To plot the correlation (`Corr()`) and RMSE (`RMS()`) between experiments 
(averaged across ensemble members) and observations:

```r
corr <- Corr(Mean1Dim(ano$ano_exp, 2), Mean1Dim(ano$ano_obs, 2))
PlotVsLTime(corr,
            toptitle = "Time correlation with observation X, over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_corr_expA_expB_obsX.eps')
```
```r
rms <- RMS(Mean1Dim(ano$ano_exp, 2), Mean1Dim(ano$ano_obs, 2))
PlotVsLTime(rms,
            toptitle = "RMSE with observation X, over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_rms_expA_expB_obsX.eps')
```
<img src="vis_corr_expA_expB_obsX.png" width="600" />
<img src="vis_rms_expA_expB_obsX.png" width="600" />
 - To plot the ratio between the RMSE of the ensemble mean of two experiments 
with a same observation at a single grid point or area averaged:

```r
ratio_rms <- RatioRMS(Mean1Dim(ano$ano_exp[1, , , ], 1),
                      Mean1Dim(ano$ano_exp[2, , , ], 1),
                      ano$ano_obs[1, 1, , ])
ratio_rms2 <- array(dim = c(1, 4, dim(ratio_rms)[2]))
ratio_rms2[1, c(2, 4), ] <- ratio_rms
PlotVsLTime(ratio_rms2,
            toptitle = "RMSE of exp. A / RMSE of exp. B (against obs. X), over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1, siglev = TRUE, leg = FALSE,
            fileout = 'vis_ratiorms_expA_expB_obsX.eps')
```
<img src="vis_ratiorms_expA_expB_obsX.png" width="600" />
 - To plot the ratio between the ensemble spread of the experiments and their 
RMSE against the observations (`RatioSDRMS()`) at a single grid point or area
averaged:

```r
ratio_sdrms <- RatioSDRMS(ano$ano_exp, ano$ano_obs)
ratio_sdrms2 <- array(dim = c(dim(ratio_sdrms)[1:2], 4, dim(ratio_sdrms)[4]))
ratio_sdrms2[, , c(2, 4), ] <- ratio_sdrms
PlotVsLTime(ratio_sdrms2,
            toptitle = "S. dev. over members and s. dates / RMSE, over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1, siglev = TRUE,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_ratiosdrms_expA_expB_obsX.eps')
```
<img src="vis_ratiosdrms_expA_expB_obsX.png" width="600" />

In this example, the ratio SD / RMS is calculated for the experiment A only 
but against two observational datasets:

```r
ratio_sdrms <- RatioSDRMS(plyr::take(ano$ano_exp, 1, 1),
                          abind::abind(ano$ano_obs, 
                                       ano$ano_obs + rnorm(length(ano$ano_obs), 0, 0.1), 
                                       along = 1))
ratio_sdrms2 <- array(dim = c(dim(ratio_sdrms)[1:2], 4, dim(ratio_sdrms)[4]))
ratio_sdrms2[, , c(2, 4), ] <- ratio_sdrms
PlotVsLTime(ratio_sdrms2,
            toptitle = "S. dev. over members and s. dates / RMSE, over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1, siglev = TRUE,
            listexp = c('Experiment A'),
            listobs = c('Observation X', 'Observation X + rnorm(n, 0, 0.1)'),
            fileout = 'vis_ratiosdrms_expA_obsX_obsXrnorm.eps')
```
<img src="vis_ratiosdrms_expA_obsX_obsXrnorm.png" width="600" />
 - To plot `RMSSS()` of ensemble mean at a single grid point or area averaged:

```r
rmsss <- RMSSS(Mean1Dim(ano$ano_exp, 2), Mean1Dim(ano$ano_obs, 2))
rmsss2 <- array(dim = c(dim(rmsss)[1:2], 4, dim(rmsss)[4]))
rmsss2[, , c(2, 4), ] <- rmsss
PlotVsLTime(rmsss2,
            toptitle = "RMSSS with observation X, over Atlantic region",
            ytitle = "K",
            monini = 12, freq = 1, siglev = TRUE,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_rmsss_expA_expB_obsX.eps')
```
<img src="vis_rmsss_expA_expB_obsX.png" width="600" />
 - To plot effective number of independent data (`Eno()`):

```r
eno <- Eno(Mean1Dim(ano$ano_exp, 2), posdim = 2)
eno2 <- array(dim = c(dim(eno)[1], 4, dim(eno)[2]))
eno2[, 2, ] <- eno
PlotVsLTime(eno2,
            toptitle = "Eff. n. of independent data, over Atlantic region",
            monini = 12, freq = 1, siglev = FALSE, show_conf = FALSE,
            listexp = c('Experiment A', 'Experiment B'),
            fileout = 'vis_eno_expA_expB.eps')
``` 
<img src="vis_eno_expA_expB.png" width="600" />

`Plot2VarsVsLTime()` allows to plot two indices or scores at a time on the same
plot, each with its confidence intervals. It accepts as inputs arrays of only 
3 dimensions:
```r
  c(n. of experiments, 3/4, n. of lead-times)
```
i.e., it accepts only indices or scores from a set of experimental datasets 
against a single observation.
Its options are identical to the ones of `PlotVsLTime()` except that the 
parameter for specifying legends for observations (`listobs`) is replaced by a 
parameter to put a legend to distinguish the plotted variables, `listvars`.
```
Plot2VarsVsLTime(corr[, 1, 1:3, ], rms[, 1, , ],
                 toptitle = "Time corr. and RMSE with observation X, over Atlantic region",
                 ytitle = "K",
                 monini = 12, freq = 1,
                 listexp = c('Experiment A', 'Experiment B'),
                 listvars = c('Corr', 'RMSE'),
                 fileout = 'vis_corr_rms_expA_expB_obsX.eps')
```
<img src="vis_corr_rms_expA_expB_obsX.png" width="600" />

`PlotACC()`, in contrast to `PlotVsLTime()`, accepts an additional dimension
for the starting dates in the input and the dimension of the confidence 
intervals is positioned at the end in place of the latitudes and longitudes, 
as provided by `ACC()`:
```r
  c(n. of experiments, n. of observational datasets, n. of starting dates, n. of lead-times, 4)
```
The ACCs and confidence intervals of all forecast times and starting dates of 
all experiments are plotted. All the starting dates of an experiment are 
painted with the same colour. There are two possible styles, switchable via 
the parameter `points`:
 - Drawing curves along forecast times, as in `PlotVsLTime()`. In that case 
`fill` can enable filling the area limited by the confidence intervals.
 - Drawing a point for each forecast time, with vertical lines joined to the
extremes of the confidence interval, limited with notches (default).
Besides, `linezero` draws a line at ordinate 0 and `vlines` draws vertical
lines at any specified set of abscissae.

```r
data <- Load('tas', list(expA, expB), list(obsX),
             sdates = sdates,
             leadtimemin = 2, leadtimemax = 13,
             latmin = -10, latmax = 60,
             lonmin = 100, lonmax = 250,
             output = 'lonlat', grid = 't106grid',
             method = 'distance-weighted')
ano <- Ano_CrossValid(data$mod, data$obs, memb = FALSE)
acc <- ACC(ano$ano_exp, ano$ano_obs)
PlotACC(acc$ACC, selected_sdates, 
        toptitle = "Spatial anom. corr. coeff. with obs X over Atlantic region",
        ytitle = "K", freq = 1,
        legends = c('Experiment A', 'Experiment B'),
        fileout = 'vis_acc_expA_expB_obsX.eps')
```
<img src="vis_acc_expA_expB_obsX.png" width="600" />

Plotting 2-dimensional fields<a name="2d_fields"></a>
-----------------------------



