Weather regime analysis
========================

s2dverification contains a set of functions to compute weather regime classifications. The goal fo this analysis is to generate a set of patterns able to characterize the different circulation structures occurring each month/season. 

In case you want to investigate the s2dverification functionalities you can:

- Load s2dverification (in particular the version including the weather regimes functions) with the following command:

```r
 devtools::install_git('https://earth.bsc.es/gitlab/es/s2dverification',
                       branch = 'develop-Regimes')
```

- Check the documentation which is available here:

https://cran.r-project.org/web/packages/s2dverification/s2dverification.pdf

### 1- Load files with the `Load ()` function included in the package. For a full description of `Load()` you can click [*here*] (https://earth.bsc.es/gitlab/es/s2dverification/blob/develop-vignettes/vignettes/data_retrieval.md).
The data employed in this example is described below.  
- Sea level pressure (psl): this has been selected as the circulation variable, however other variables such as geopotential at 500 hPa can be also used.
- Region: traditional Euro-Atlantic domain [85.5ºW-45ºE; 27-81ºN]]
- Datasets: seasonal predictions from ECMWF System 4 ([**Molteni et al. 2011**](https://www.ecmwf.int/sites/default/files/elibrary/2011/11209-new-ecmwf-seasonal-forecast-system-system-4.pdf) and ERA-Interim reanalysis ([**Dee et al. 2011**](http://onlinelibrary.wiley.com/doi/10.1002/qj.828/pdf)) as a reference dataset.
- Period: for illustration we have only selected 20 years (1990-2010), but the full period available can be employed (1981-2017). 

```r
sdates <- paste0(1990:2010, '1201')
data <- Load('psl', exp = 'system4_m1',
             obs = 'erainterim',
             sdates = sdates,
             storefreq='daily',
             leadtimemin = 1, leadtimemax = 31,
             latmin = 27, latmax = 81,
             lonmin = 274.5, lonmax = 45,output = 'lonlat')
```
The object returned by `Load()` contains, among others, two arrays with the 
requested data, one for the experimental data and the other for observational 
data (`data$obs` and `data$mod`).

### 2- Daily anomalies based on a smoothed climatology
The weather regimes classification is based on daily anomalies, which have been computed by following these steps: 

- Daily climatology is computed with the `Clim()` function.

```r
clim<-Clim(var_exp = data$mod,var_obs = data$obs,memb=F)
```
- A LOESS filter has been applied to the climatology to remove the short-term variability and retain the annual cycle and to better isolate quasi stationary periods. The parameter `loess_span` should be adapted to the period used to compute the climatology (e.g. season, month, week,...).

```r
Loess<-function(clim,loess_span){
  data<-data.frame(ensmean=clim,day=1:length(clim))
  loess_filt<-loess(ensmean~day,data,span=loess_span)
  output<-predict(loess_filt)
  return(output)
}
clim_smoothed_obs<-aperm(apply(clim$clim_obs,c(1:length(dim(clim$clim_obs)))[-which(names(dim(clim$clim_obs))=='ftime')],Loess,loess_span=1),c(2,1,3,4))
clim_smoothed_exp<-aperm(apply(clim$clim_exp,c(1:length(dim(clim$clim_exp)))[-which(names(dim(clim$clim_exp))=='ftime')],Loess,loess_span=1),c(2,1,3,4))
```
- Anomalies are computed with the `Ano()` function, as the deviations of the sea level pressure from the smoothed climatologies.

```r
anom_obs<-Ano(data$obs,clim_smoothed_obs)
anom_exp<-Ano(data$mod,clim_smoothed_exp)
```

### 3- Weather regimes in observations 

`WeatherRegimes()` function is used to define the clusters corresponding to the ERA-Interim classification. There are different assumptions that should be performed to obtain the weather regimes. In this example these assumptions are: 

-  k-means algorithm has been employed to produce a partition of all states in a predefined number of clusters (`ncenters`). These partitions maximizes the ratio of variance among the clusters centroids and the intra-cluster variance. In this example we have selected four partitions, which is the standard in the literature.  
- Cluster partitions are obtained from an iterative process (default `iter.max=100`) in which the centroid coordinates are those that produces the maximum variance ratio. In this example the 30 initial random centroid coordinates have been used (default, `nstart = 30` ).
- To reduce the dimensionality of the problem EOF can be used, however in this example we have not computed EOFs just to take into account the extreme values (`EOFS=FALSE`). 
- Anomalies introduced in the k-means function should be latitude weighted (`lat_weights=TRUE`), therefore latitude values are introduced as input (`lat` input).

```r
WR_obs<-WeatherRegime(data=anom_obs,EOFS=FALSE,lat_weights = TRUE,lat=data$lat,ncenters = 4)
```

`WeatherRegime()` provides a list with 6 elements (`$composite`, `$pvalue`, `$cluster`,`$center`, `$cluster_length`, `$cluster_values`, `$persistence` and `$frequency`) which are the needed parameters for the weather regimes assessment. Further details about the output provided by the `WeatherRegime()` function can be found in the package documentation (link WeatherRegime() documentation). 

### 4- Visualisation of the observed weather regimes

To plot the composite maps of each regime and the mean frequencies of each cluster, we have employed the `PlotLayout()` and `PlotEquiMap()` functions available in s2dverifcation. The object `WR_obs$composite` is divided by 100, just to reperesent the anomalies in hPa. 

 ```r
titles<-paste0('freq = ',round(WR_obs$frequency,1),'%')
PlotLayout(PlotEquiMap,c(1,2),lon=data$lon,lat=data$lat,var=WR_obs$composite/100,
           titles=paste0(paste0('Cluster ',1:4),' (',paste0('freq = ',round(WR_obs$frequency,1),'%'),' )'),filled.continents=F,
           axelab=F,draw_separators = T,subsampleg = 1,brks=seq(-16,16,by=2),
           bar_extra_labels = c(2,0,0,0),fileout='observed_regimes.png')
```

<img src="observed_regimes.png" />


### 5- Weather regimes in the predictions

Predicted anomalies for each day,month,member and lead time are matched with the observed clusters (obtained in step 3). The map of anomalies is assigned to the particular cluster for which the Eucledian distance `method='distance'` reaches the minimum value (The matching can also be also done in terms of spatial correlation `method='ACC'`, however the distance method is more computationally efficient than the ACC method). The assignation of the anomalies to a pre-defined set of clusters guarantees that the predicted weather regimes have very similar spatial structures to the observed regimes, which is essential for the verification of the predicted weather regimes. 

 ```r
reference<-drop(WR_obs$composite)
names(dim(reference))<-c('lat','lon','nclust')
WR_exp<-RegimesAssign(var_ano=anom_exp,ref_maps=reference,lats=lats)
```
`RegimesAssign()` provides a list with 4 elements (`$composite`, `$pvalue`, `$cluster` and `$frequency`) with similar information to that provided by the `WeatherRegime()` function. Further details about the output can be found in the package documentation (link RegimesAssign() documentation). 

### 6- Visualisation of the predicted weather regimes

The outputs of `RegimesAssign()` have been represented to visually compare predicted and observed weather regimes. 

```
PlotLayout(PlotEquiMap,c(1,2),lon=data$lon,lat=data$lat,var=WR_exp$composite/100,
           titles=paste0(paste0('Cluster ',1:4),' (',paste0('freq = ',round(WR_exp$frequency,1),'%'),' )'),
           filled.continents=F,
           axelab=F,draw_separators = T,subsampleg = 1,brks=seq(-16,16,by=2),
           bar_extra_labels = c(2,0,0,0),fileout='predicted_regimes.png')
```
<img src="predicted_regimes.png" />

