---
author: "An-Chi Ho"
date: "`r Sys.Date()`"
output: html_document
vignette: >
  %\VignetteEngine{R.rsp::md}
  %\VignetteIndexEntry{Visualisation}
  %\usepackage[utf8]{inputenc}
---

# Visualisation Introduction

The plotting functions can be used at all stages of the verification process.
These functions are essential to:
 - Quickly inspect the results of a newly produced experiment, i.e. to check 
the physical consistency of the results.
 - Assess the added value of a new prediction system, i.e. to compare new 
 results with a reference (observation, reconstruction or other experiment).
 - Assess visually the significance of results, i.e. to display in a 
user-friendly way confidence intervals and other statistics.

The visualisation functions, most with a name following the pattern 
`Viz*()`, can be grouped into the kind of plot they provide: 
#TODO: Add link
 - **Plotting time series**: `VizClim()`, `VizAno()`, 
`VizVsLtime()`, `Viz2VarsVsLTime()`, `VizBoxWhisker()` and `VizACC()`.
 - **Plotting maps**: `VizEquiMap()`, `VizRobinson()`,
`VizStereoMap()`, `AnimateMap()`, `VizLayout()` and `VizSection()`.

To exihibit the usage of the plotting functions, we need to have data loaded first.
Here, we use startR package to load the experimental and observational data from our data archive (esarchive).
If you don't have access to esarchive, you can contact package maintainer (an.ho@bsc.es; eva.rifarovira@bsc.es)
to obtain the sample data. 
Next, we will go through each plotting function and explain of which situations they fit the best, 
also details of the options they provide, and short examples of usage.

## Data loading

We use two datasets: the experiment data are SEAS5 from ECMWF, and the observation ones are ERA5 from ECMWF.

```r
library(startR)

sdates <- paste0(1997:2016, '1101')

repo_exp1 <- paste0('/esarchive/exp/ecmwf/system5c3s/monthly_mean/',
                    '$var$_f6h/$var$_$sdate$.nc')
repo_exp2 <- paste0('/esarchive/exp/meteofrance/system7c3s/monthly_mean/',
                    '$var$_f6h/$var$_$sdate$.nc')

exp <- Start(dat = list(list(name = "system5c3s", path = repo_exp1),
                        list(name = "meteofrance", path = repo_exp2)),
             var = 'tas',
             sdate = sdate_exp,
             time = 1:6,
             ensemble = 'all',
             lat = values(list(20, 80)),
             lat_reorder = Sort(),
             lon = values(list(-20, 40)),
             lon_reorder = CircularSort(-180, 180),
             transform = CDORemapper,
             transform_params = list(grid = 'r360x181', method = 'bilinear'),
             transform_vars = c('lat', 'lon'),
             return_vars = list(lat = 'dat', lon = 'dat',
                                time = 'sdate'),
             retrieve = TRUE)

# Use exp time metadata to load obs
sdate_obs <- format(attributes(exp)$Variables$common$time, "%Y%m")
dim(sdate_obs) <- dim(attributes(exp)$Variables$common$time)

repo_obs <- '/esarchive/recon/ecmwf/era5/monthly_mean/$var$_f1h-r1440x721cds/$var$_$sdate$.nc'

obs <- Start(dat = repo_obs,
             var = 'tas',
             sdate = sdate_obs,
             split_multiselected_dims = TRUE,
             lat = values(list(20, 80)),
             lat_reorder = Sort(),
             lon = values(list(-20, 40)),
             lon_reorder = CircularSort(-180, 180),
             transform = CDORemapper,
             transform_params = list(grid = 'r360x181', method = 'bilinear'),
             transform_vars = c('lat', 'lon'),
             synonims = list(lat = c('lat', 'latitude'),
                             lon = c('lon', 'longitude')),
             return_vars = list(time = 'sdate',
                                lon = NULL, lat = NULL),
             retrieve = TRUE)
```

Calculate spatial mean.

```r
lat <- attributes(exp)$Variables$system5c3s$lat
lon <- attributes(exp)$Variables$system5c3s$lon

exp_sm <- ClimProjDiags::WeightedMean(exp, lat = lat, lon = lon, latdim = "lat", londim = "lon", na.rm = T)
dim(exp_sm) <- dim(exp_sm)[c("dat", "sdate", "time", "ensemble")]

obs_sm <- ClimProjDiags::WeightedMean(obs, lat = lat, lon = lon, latdim = "lat", londim = "lon", na.rm = T)
dim(obs_sm) <- dim(obs_sm)[c("dat", "sdate", "time")]
```


## Plotting maps

This group of functions allows to plot grid data (i.e. defined over latitudes 
and longitudes) on a rectangular equidistant projection or on a stereographic 
projection as well as depth sections (i.e. defined over latitudes/longitudes and depth levels).

Regarding the functions to plot maps, by default each grid point is drawn on a 
world map with a colour as a function of the magnitude of the provided field, 
index or score at that point. While `VizEquiMap()` and `VizStereoMap()` plot 
data of a single forecast time, `AnimateMap()` creates an animation with the 
plotted maps of all forecast times obtained with one of the previous functions at 
choice.

`VizEquiMap()` and `VizStereoMap()` share some traits:
 - Both expect a data matrix in the parameter `var` of dimensions 
`c(n. of latitudes, n. of longitudes)`, `lat` with a vector with the latitudes 
of the centers of the grid cells and `lon`, a vector with the longitudes. 
`Load()` already provides these in the correct format, through the components 
`$lon` and `$lat`.
 - The N colours to paint the grid cells with (via `cols`) as well as the N +
1 threshold magnitudes (via `brks`) that will allow to assign each grid cell 
value one of the N colours. The colour for any missing values can be adjusted 
with `colNA`.
 - Whether to fill the map continents or only draw coast wires can be adjusted
with `filled.continents`.
 - A matrix of `dots` can be plotted over the drawn map to, e.g., highlight 
cells where a skill has been greatest or most significant.
 - A legend can be drawn (unless disabled with `drawleg`) and the amount of 
ticks on that legend can be adjusted with `subsampleg`.
 - They accept any additional parameters via the parameter `...` to be sent to
the underlying R graphics `image()` function for a fine tuning.

### VizEquiMap()

Next some examples of maps obtained with `VizEquiMap()`:
```r
VizEquiMap(s2dv::MeanDims(map_data$mod, 2)[1, 1, 1, , ], map_data$lon, map_data$lat,
            toptitle = "Exp. A 'tas', s. date: 1990-11-01, f. time: 1 month", 
            units = "K", fileout = 'vis_equimap_raw_expA.png')

VizEquiMap(s2dv::MeanDims(map_data$obs, 2)[1, 1, 1, , ], map_data$lon, map_data$lat,
            toptitle = "Obs. X: 'tas', 1990-12-01", 
            units = "K", filled.continents = FALSE, fileout = 'vis_equimap_raw_obsX.png')
```
<img src="./Figures/vis_equimap_raw_expA.png" width="600" />
<img src="./Figures/vis_equimap_raw_obsX.png" width="600" />

```r
VizEquiMap(s2dv::MeanDims(map_data$mod, 2)[1, 1, 1, , ], map_data$lon, map_data$lat,
            toptitle = "Exp. A 'tas', s. date: 1990-11-01, f. time: 1 month", 
            units = "K", brks = 100, bar_limits = c(255, 300),
            filled.continents = 'black',
            fileout = 'vis_equimap_cols_raw_expA.png')

VizEquiMap(s2dv::MeanDims(map_data$obs, 2)[1, 1, 1, , ], map_data$lon, map_data$lat,
            toptitle = "Obs. X 'tas', 1990-12-01", 
            units = "K", brks = 100, bar_limits = c(255, 300),
            filled.continents = FALSE, 
            fileout = 'vis_equimap_cols_raw_obsX.png')
```
<img src="./Figures/vis_equimap_cols_raw_expA.png" width="600" />
<img src="./Figures/vis_equimap_cols_raw_obsX.png" width="600" />


Or, as seen in the example from [**Snippet 2**](snippets.md#snippet2):<a name="snippet2"></a>

<img src="./Figures/snip2_equimap_corr_raw_expA_obsX.png" width="600" />
<img src="./Figures/snip2_equimap_corr_raw_expB_obsX.png" width="600" />


`VizEquiMap()` has some other additional features:
 - Drawing contour lines: `square` allows to smooth the borders of each 
coloured region and draws contour lines and figures. The contours can be 
adjusted by providing a matrix to `contours` with dimensions 
`c(n. of longitudes, n. of latitudes)`. The contours of the data in that 
matrix will be drawn over the map, using the thresholds provided in `brks2`.
This is useful to plot the contours of the original data without border 
smoothing, to plot only some of the default contours or to plot other contours 
defined by another field.  

```r
VizEquiMap(s2dv::MeanDims(map_data$mod, 2)[1, 1, 1, , ], map_data$lon, map_data$lat,
            toptitle = "Exp. A 'tas', s. date: 1990-11-01, f. time: 1 month", 
            units = "K", brks = 100, bar_limits = c(255, 300), square = FALSE,
            fileout = 'vis_equimap_contour_raw_expA.png')

VizEquiMap(s2dv::MeanDims(map_data$obs, 2)[1, 1, 1, , ], map_data$lon, map_data$lat,
            toptitle = "Obs. X 'tas', 1990-12-01", 
            units = "K", brks = 100, bar_limits = c(255, 300), 
            square = FALSE, contour_lty = 2,
            fileout = 'vis_equimap_contour_raw_obsX.png')
```

<img src="./Figures/vis_equimap_contour_raw_expA.png" width="600" />
<img src="./Figures/vis_equimap_contour_raw_obsX.png" width="600" />

 - Drawing boxes on the map: `boxlim`, `boxcol` and `boxlwd` allow to 
specify the position of the corners, colour and thickness of a box to be drawn 
on the map. 

```r
VizEquiMap(s2dv::MeanDims(map_data$obs, 2)[1, 1, 1, , ], map_data$lon, map_data$lat,
            toptitle = "Obs. X 'tas', 1990-12-01", 
            units = "K", brks = 100, bar_limits = c(255, 300),
            boxlim = list(c(-62, -22, -23, 15), c(-30, 10, 0, 32)), 
            boxcol = c('purple', 'blue'), boxlwd = c(8, 4),
            coast_color = 'black',
            fileout = 'vis_equimap_box_expA.png')
```

<img src="./Figures/vis_equimap_box_expA.png" width="600" />

 - Ticks on the longitude/latitude axes can be adjusted with `axelab`, `labW`, 
`intylat` and `intxlon`.
 - `numbfig` allows to specify the number of figures in the final layout in 
order to adjust some spacing parameters.

### VizStereoMap()

`VizStereoMap()` can be called almost identically to `VizEquiMap()`, except
that it requires the `latlims` parameter to specify which range of latitudes to
be plotted on the stereographic projection map.  
The only special parameter is the `intlat`, to set the interval in degrees 
between consecutive latitude circles on the plot.

Next a couple of examples:

```r
## Loading data over the entire globe
world_data <- Load('tas', list(expA, expB), list(obsX),
                    sdates = sdates[1],
                    leadtimemin = 2, leadtimemax = 13,
                    latmin = -90, latmax = 90,
                    lonmin = 0, lonmax = 360,
                    output = 'lonlat', grid = 't106grid',
                    method = 'distance-weighted')
VizStereoMap(s2dv::MeanDims(world_data$mod, 2)[1, 1, 1, , ], 
              world_data$lon, world_data$lat, c(40, 90),
              brks = 100, bar_limits = c(240, 290),
              toptitle = "Exp. A 'tas', s. date: 1990-11-01, f. time: 1 month",
              title_scale = 0.5,
              units = "K", filled.continents = 'black',
              fileout = 'vis_stereomap_raw_expA.png')

VizStereoMap(s2dv::MeanDims(world_data$obs, 2)[1, 1, 10, , ], 
              world_data$lon, world_data$lat, c(-90, -60), intlat = 10, 
              brks = 100, bar_limits = c(240, 290),
              boxlim = list(c(0, -85, 135, -70)),
              toptitle = "Obs. X 'tas', 1990-12-01",
              title_scale = 0.5,
              units = "K", fileout = 'vis_stereomap_raw_obsX.png')
```

<img src="./Figures/vis_stereomap_raw_expA.png" width="600" />
<img src="./Figures/vis_stereomap_raw_obsX.png" width="600" />

### AnimateMap()

`AnimateMap()` generates animations of maps from `VizEquiMap()` or 
`VizStereoMap()` and saves them in GIF files at the paths specified in the
parameter `fileout`. It receives as main parameter, similarly as 
`VizVsLTimes()`, an array with the dimensions:
```r
  c(n. of experiments/observations, n. of observations, 3, n. of forecast times, n. of latitudes, n. of longitudes)
```
the first 3 being optional and the 3rd dimension corresponding to the lower 
confidence interval, the actual value and the upper confidence interval. This 
means it can plot maps of a field of one or multiple experimental or 
observational datasets, averaged across ensemble members and starting dates 
(climatologies) or indices or scores computed across starting dates against 
one or more observations. The confidence intervals, if provided, are used to 
draw black dots on grid cells that reach a 95% significance level, if 
requested via `msk95lev`.

The provided data is assumed to be in a monthly frequency and to start at 
January. Otherwise it can be adjusted with `monini` and `freq`.

Whether to plot with a equidistant rectangular projection or a stereographic 
projection can be chosen with the parameter `equi`. The plot can be limited to 
a sub-region of the provided data via the parameters `lonmin`, `lonmax`, 
`latmin` and `latmax`.

`AnimateMap()` allows for the typical adjustments in the map plotting 
functions: specifying the palette colors and breaks (`col` and `brk`),
displaying or not a legend (`drawleg`, `subsampleg`), selecting a color for
the NA values (`colNA`) and filling or not the continents 
(`filled.continents`).

Further title and axis tick adjustments can be achieved with `toptitle`, 
`sizetit`, `units`, `intlon` and `intlat`.
brk col drawleg subsampleg colNA filled.continents

Next a few examples:

```r
map_clim <- s2dv::Clim(map_data$mod, map_data$obs, memb = FALSE)

cols <- ClimColors(50)

data_min <- min(map_data$mod, map_data$obs, na.rm = TRUE)
data_max <- max(map_data$mod, map_data$obs, na.rm = TRUE)
brks <- round(seq(data_min, data_max, length.out = length(cols) + 1), 2)

AnimateMap(ClimProjDiags::Subset(map_clim$exp, 'dataset', 1), 
           map_data$lon, map_data$lat, monini = 12, 
           toptitle = "Exp. A climatologies.", 
           units = "K", brks = brks, cols = cols,
           fileout = "vis_anim_clim_expA.gif")
```
<img src="./Figures/vis_anim_clim_expA.gif" width="600" />

And, as seen in [**Snippet 2**](snippets.md#snippet2), the animations of the 
actual time correlations of Experiment A and B against Observation X over the 
Atlantic, with black dots on values that reach a 95% significance level:

<img src="./Figures/snip2_anim_corr_expA_obsX.gif" width="600" />

<img src="./Figures/snip2_anim_corr_expB_obsX.gif" width="600" />

Also the entire globe and stereographic projection maps can be animated:

```r
world_clim <- s2dv::Clim(world_data$mod, world_data$obs, memb = FALSE)
AnimateMap(ClimProjDiags::Subset(world_clim$exp, 'dataset', 1), 
           world_data$lon, world_data$lat, 
           filled.continents = FALSE, monini = 12,
           toptitle = "Exp. A climatologies.", units = "K",
           brks = brks, cols = cols, 
           fileout = 'vis_anim_clim_expA_world.gif')
AnimateMap(world_clim$obs, 
           world_data$lon, world_data$lat, equi = FALSE,
           latmin = 60, latmax = 90, monini = 12,
           toptitle = "Obs. X climatologies.", 
           units = "K", brks = brks, cols = cols,
           fileout = "vis_anim_clim_obsX_world.gif")
```

<img src="./Figures/vis_anim_clim_expA_world.gif" width="600" />

<img src="./Figures/vis_anim_clim_obsX_world.gif" width="600" />

### VizLayout()

This function allows to easily combine plots from any R plot function in 
layouts. The input data can be provided in multiple data arrays, and a common
colour bar can be set for the multi-panel.

A common use case is to plot the ensemble members at a given time step:

```r
VizLayout(VizEquiMap, c('lat', 'lon'),
           ClimProjDiags::Subset(map_data$mod, 
                  list('dataset', 'sdate', 'ftime'), 
                  list(1, 1, 1)), 
           map_data$lon, map_data$lat, units = 'K',
           toptitle = "Exp. A 'tas', s. date: 1990-11-01, f.time: 1 month",
           titles = paste("Member", 1:dim(map_data$mod)[2]),
           brks = 50, bar_limits = c(250, 300), coast_color = 'black',
           fileout = "vis_layout_equimap_expA.png") 
```

<img src="./Figures/vis_layout_equimap_expA.png" width="600" />

But really complex layouts can be achieved thanks to the great number of
available parameters:

```r
ens_mean <- ClimProjDiags::Subset(world_data$mod, 
                   list('dataset', 'sdate', 'ftime'), 
                   list(1, 1, 1))
layout <- VizLayout(fun = c('VizEquiMap', 'plot', 'plot', 'VizStereoMap'),
                     plot_dims = c('lat', 'lon'), 
                     var = list(ens_mean, array(1:10), 
                                array(10:1), ens_mean), 
                     lon = world_data$lon, lat = world_data$lat, 
                     fill = 'black', sizetit = 0.5, axes_label_scale = 0.6, 
                     coast_color = 'yellow',
                     titles = paste('Fig.', 1:12), toptitle = 'Multipanel', 
                     row_titles = paste('Row', 1:3), col_titles = paste('Col', 1:4), 
                     drawleg = 'E', units = 'K', units_scale = 2, 
                     bar_limits = c(275, 300), brks = 26, 
                     bar_extra_labels = 281:284, 
                     width = 12, height = 10, res = 200, 
                     fileout = 'vis_layout_complex.png')
```

<img src="./Figures/vis_layout_complex.png" width="800" />

### VizSection()


