Tutorial
========

Seasonal forecast of extreme events
-----------------------------------

### Objectives
- Basics of R language
- Learning to use s2dverification

### 1- Load files, compute basic statistics, plot map and time series

#### Exercise 1 - First steps with R and s2dverification
- Make sure you have CDO tools installed in your system, or download and install
as described in:

https://code.zmaw.de/projects/cdo/wiki/cdo

- Open R (typing R in the terminal).
- Install s2dverification with:

```r
install.packages('s2dverification')
```

- Load s2dverification with the following command:

```r
library(s2dverification)
```

- The documentation of s2dverification is available online here:

https://cran.r-project.org/web/packages/s2dverification/s2dverification.pdf

- You can see the list of available functions in the package by typing:
```r
help(package = s2dverification)
```
- To see the help of a specific function, you can type:

```r
help(FunctionName)
```

- Here I give you an example of how to use the `Load()` function. It assumes 
you have some *tas* data for the experiments from all the ENSEMBLES models 
(named *EnsCmccSeas*, *EnsIfmSeas*, *EnsEcmwfSeas*, *EnsMetfrSeas* and 
*EnsUkmoSeas* at BSC-ES) and for the ERA-interim reanalysis (*ERAint*) in the 
folders */example/experiments* and */example/observations*.
See the [**Example**](example.md) of use or 
[**Data retrieval**](data_retrieval.md) for a full explanation of what is done 
here. 

```r
exp_path_pattern <- file.path("/example/experiments/ENSEMBLES/$EXP_NAME$",
                              "$STORE_FREQ$_mean/$VAR_NAME$",
                              "$VAR_NAME$_$START_DATE$.nc")
cmcc_seas <- list(name = "EnsCmccSeas", path = exp_path_pattern)
ifm_seas <- list(name = "EnsIfmSeas", path = exp_path_pattern)
ecmwf_seas <- list(name = "EnsEcmwfSeas", path = exp_path_pattern)
metfr_seas <- list(name = "EnsMetfrSeas", path = exp_path_pattern)
ukmo_seas <- list(name = "EnsUkmoSeas", path = exp_path_pattern)

obs_path_pattern <- file.path("/example/observations/$OBS_NAME$",
                              "$STORE_FREQ$_mean/$VAR_NAME$",
                              "$VAR_NAME$_$YEAR$$MONTH$.nc")
era_int <- list(name = "ERAint", path = obs_path_pattern)

library(s2dverification)
Data <- Load("tas", 
             exp = list(cmcc_seas, ifm_seas, ecmwf_seas, metfr_seas, ukmo_seas),
             obs = list(era_int),
             sdates = c("19930501", "19940501"), 
             leadtimemin = 1, leadtimemax = 7, storefreq = "monthly", 
             sampleperiod = 1, nmember = 9, output = "areave", 
             lonmin = 190, lonmax = 240, latmin = -5, latmax = 5)
```

- You can copy paste it in R, run and check everything is working fine. Once it is finished have a look at the dimension of `Data` by typing this command in R:
```
dim(Data$mod)
dim(Data$obs)
```

- With the help of the `Load()` function try to undestand what the different 
dimensions are.
- Now run another time `Load()` with the same parameters, but change 
`output = 'areave'` to `output = 'lonlat'`.
- What are the dimensions of `Data` now?
- What happens if you remove the `lonmin`, `lonmax`, `latmin`, `latmax` 
options?
- What happens if you change the `leadtimemin` and `leadtimemax` options?

#### Exercise 2 - First compute and plot 2m-temperature skill over Europe

- In the directory named *handson* create a directory named *R* and browse into
that directory.
- Open a new file named *corrskill-europe.R*.
- At the beginning of the file, load the s2dverification package:

```r
library(s2dverification)
```

- If you are using emacs and you don't have colour highlighting just execute 
this line in terminal and reopen the file:

```
/usr/local/bin/ictp-install ess
```

- Generate a list of starting dates to be used in the `sdates` argument of 
`Load()` which includes all May starting dates between 1979 and 2005 (you can 
have a look at the help of the function `seq` and `paste` to do it).
- Based on the example of the previous exercise, try to load the lead-times 2, 
3 and 4 of 2m-temperature (*tas*) of all ENSEMBLES models, over Europe 
(20W70E-25N75N) for all May starting dates between 1979 and 2005.
- Calculate the June-July-August seasonal mean using the `Mean1Dim()` function
for both models and observations.
- With the same function, calculate the ensemble mean of each model.
- With the `Corr()` function, calculate the time correlation in 
June-July-August between each model and the ERA-interim reanalysis.
- Using `PlotEquiMap()`, plot a map of the correlation coefficient for each 
model and save it in a postscript (You can use the `postscript()` and 
`dev.off()` functions). You can choose the following interval and the following
color bar:

```r
colors <- clim.colors(20)
interval <- seq(-1, 1, length.out = length(colors) + 1)
```

or generate another one using the `brewer.pal()` function.
- If you want to generate a multipanel plot you can use the `layout()` 
function (you can have a look at this document for more help: 
http://seananderson.ca/courses/11-multipanel/multipanel.pdf). Be carefeul, 
some features of `PlotEquiMap()` are not available with multipanel plots.

#### Exercise 3 - Compare the skill over land over the mediterranean region

- In the directory named *handson/R*, open a new file named 
*Timeserie-skill-mediter.R*.
- The mediterranean region 3E25E-36N44N covers both sea and land, but we would
like to calculate the skill only over land. For this, you need to use a 
have the land-sea masks of ENSEMBLES and ERA-interim in NetCDF files, for 
example, */path/to/experiments/ENSEMBLES/masks/land_sea.nc* and 
*/path/to/observations/ERAint/masks/land_sea.nc*.
- To create masks usable in the `maskobs` and `maskmod` arguments of the 
`Load()` function, you can use the following lines:

```r
fnc <- nc_open('/path/to/experiments/ENSEMBLES/masks/land_sea.nc')
maskERAI <- ncvar_get(fnc, 'LSM')
nc_close(fnc)
maskERAI[which(is.na(maskERAI))] <- 1

fnc <- nc_open('/path/to/observations/ERAint/masks/land_sea.nc')
maskENS <- ncvar_get(fnc, 'LSM')
nc_close(fnc)
maskENS[maskENS > 0.5] <- 1
maskENS[maskENS <= 0.5] <- 0

listmaskmod <- list(maskENS, maskENS, maskENS, maskENS, maskENS)
```

- Use the `Load()` function with `maskobs` and `maskmod` arguments to load the
data averaged over the land in the mediterranean region for all ENSEMBLES 
models and ERA-interim reanalisys, for all May starting dates between 1979 and
2005.
- With `Clim()` and `Ano()` functions of s2dverification, calculate the 
anomalies of both models and observation.
- Calculate the ensemble mean of the models.
- Calculate the Root Mean Square Skill Score (RMSSS) for all lead-times, with 
the RMSSS function of s2dverification.
- Use the `PlotVsLTime()` function to plot this score and save it in a 
PostScript file. `PlotVsLTime()` expects a third dimension of the score of 
size 4 (lower confidence interval, score, upper confidence interval and 
significance level), while with `RMSSS()` you will have only 2 values (score
and p-value). To do it you can use the following commands to reshape your 
score matrix:

```r
skillreshape <- array(dim = c(5, 1, 4, 7))
skillreshape[, , c(2, 4), ] <- skill[, , c(1, 2), ]
```

- Do the same plot, but instead of plotting a line for the p-values, mark with 
a dot the significance values (p-values under 0.05). To do this you can have a
look at the tutorial at

http://www.harding.edu/fmccown/r/
