# Use case 0.1: Loading a sample dataset

Author: Victòria Agudetse Roures

## Goal
Load a sample dataset into SUNSET without the need configure an archive of netCDF or GRIB files.

## 0. SUNSET software stack

The first step to use SUNSET is to create a copy of the code in your local environment. Open a terminal and `cd` to the directory where you would like to store your local copy of SUNSET. For example: `/esarchive/scratch/<your_username>/git/`. If a directory does not exist yet, you can create it with the `mkdir` shell command.

```shell
# Clone the GitLab repository to create a local copy of the code
git clone https://earth.bsc.es/gitlab/es/sunset.git
```

If you are using BSC infrastructure, all of the software dependencies are already installed within our common environment. However, if you are running SUNSET outside of the BSC facilities, you can install the dependencies by creating a conda environment. A script is provided in the SUNSET repository, and you can install the environment by running the following line from the main folder of the SUNSET repository:

```shell
bash conda_installation/load_sunset.bash <conda_dir>
```

To run the line above, you should replace `<conda_dir>` with the path where you want the conda environment to be installed. For example, `/home/<user>/conda-sunset`.

## 1. The recipe

SUNSET uses YAML configuration files called 'recipes' to specify which data you want to load and the details of the different steps of the workflow. In this example, we are using a sample dataset which contains data of temperature-at-surface (tas) monthly means, from ECMWF SEAS5 as the experiment and ERA5 as the reference dataset, for the initialization month of November.

There is a template file for this hands-on tutorial, which you can open with a text editor:

```shell
# cd to the main SUNSET directory
# Open the recipe with a text editor such as vim or emacs
vim use_cases/ex0_1_sample_dataset/ex0_1-recipe.yml
```

Once you have opened your recipe, it is time to edit the contents. Fill in the blank slots according to your preference, based on the options given in the description. 

NOTE: In this particular example, the sample data is limited, so the system and reference names, variable, forecast times and region cannot be changed.

```yaml
Description:
 Author: <___>
 Description: Exercise 0.1: Skill assessment of a sample dataset
Analysis:
 Horizon: Seasonal
 Variables:
   name: tas
   freq: monthly_mean
   units: <___> # Choose your units: C or K
 Datasets:
   System:
     name: ECMWF-SEAS5.1
     Multimodel: no
   Reference:
     name:  ERA5
 Time:
   sdate: '1101'
   fcst_year: 
   hcst_start: '2000'
   hcst_end: '2006'
   ftime_min: 1
   ftime_max: 3
 Region:
   latmin:  27
   latmax:  48
   lonmin:  0
   lonmax:  359
 Regrid:
   method:  bilinear
   type:  'to_system'
 Workflow:
   Anomalies:
     compute: yes
     cross_validation: yes
     save: 'none'
   Calibration:
     method: <___>
     save: 'none'
   Skill:
     metric: RPSS BSS10 BSS90
     cross_validation: yes
     save: 'none'
   Probabilities:
     percentiles: [[1/3,  2/3]]
     save: 'none'
   Visualization:
     plots: skill_metrics
     multi_panel: no
     projection: cylindrical_equidistant
   ncores: 10
   remove_NAs: yes
   Output_format: S2S4E
Run:
  filesystem: sample # This parameter specifies that we want to load the sample data, rather than data from a filesystem
  Loglevel: INFO
  Terminal: yes
  output_dir: <______> # Path to the directory where you want your outputs to be saved
  code_dir: <______> # Path to the directory where your code is
```

## 2. Load the required SUNSET modules and read the recipe

If you are running SUNSET within BSC infrastructure, source the MODULES file to load the environment modules needed to run SUNSET.

```shell
source MODULES
```

Otherwise, activate the conda environment:

```shell
conda activate <conda_dir>
```

Open an R session, by simply typing `R` on the terminal.

To run SUNSET, we must run the R session from the directory where the code is. To check your working directory, you can run the shell command `pwd`. From the R session, you can use the commands `getwd()` and `setwd()` to see and change your working directory.

```r
# Load required modules
source("modules/Loading/Loading.R")
source("modules/Units/Units.R")
source("modules/Calibration/Calibration.R")
source("modules/Anomalies/Anomalies.R")
source("modules/Skill/Skill.R")
source("modules/Saving/Saving.R")
source("modules/Visualization/Visualization.R")

# Read recipe
recipe_file <- "use_cases/ex0_1_sample_dataset/ex0_1-recipe.yml"
recipe <- prepare_outputs(recipe_file)
```

The function `prepare_outputs()` creates a unique folder for the logs, data files and plots that result from the execution of your recipe, inside the directory you specified.  It also runs a check over the recipe to detect any potential errors, misspellings or missing arguments. At the end of the check, a message is displayed indicating whether or not the recipe passed the check, along with the list of errors and warnings.

**Questions**

Read the logs!

(1) Did your recipe pass the check? Did you get any warnings?

(2) Where will your outputs be saved? Copy and paste this directory somewhere, so that you can check it later!

*Tip*: The recipe is now stored as a `list` containing all the information of the original YAML file, plus some extra things! If you want to see any particular element of the recipe from the R session, you can simply access that element in the list. For example: 

```r
# Checking the variable name
recipe$Analysis$Variables$name
# Checking the output directory
recipe$Run$output_dir
```

## 3. Load the data and change the units

The **Loading** module retrieves the information from the recipe to load the data that has been requested it in. It loads the experiment data for the hindcast period, the reference data for the corresponding period, and the experiment forecast if a forecast year has been requested. In the case of the sample dataset, we have a hindcast and the corresponding observations, but no forecast.

For certain variables like temperature, precipitation or sea level pressure, the user can request for specific units to load the data in. The **Units** module will read the original units as stored in the netCDF files and perform any necessary unit converstions to match the request in the recipe. It also verifies that all of the loaded datasets share the same units, even if no specific unit has been requested. For this reason, users are strongly encouraged to run it even if they did not request any unit conversion.

```r
# Load datasets
data <- Loading(recipe)
# Change units
data <- Units(recipe, data)
```

**Questions**

(1) What is the structure of `data`? What is the class of the objects in `data`? *Tip*: you can use functions like `class()`, `names()` or `str()` to gain information about the structure of the object and its contents.

```r
class(data)
names(data)
str(data, max.level = 2)
# You can access any of the three objects with the `$` operator:
class(data$hcst)
```

(2) Pay attention to the log messages: Did your units get converted? Are the new units what you expect? You can check the metadata of any of the objects in data. SUNSET also provides the `data_summary()` function, which lets you have a quick look at your objects:

```r
# Check the new units and data of the hindcast (hcst) and/or observations (obs). Are they the same?
data$hcst$attrs$Variable$metadata$tas$units
data_summary(data$hcst, recipe)
data_summary(data$obs, recipe)
```
(3) What are the dimensions of the datasets? Are they consistent with what is requested in the recipe? *Tip*: Check the data summary!

## 4. Calibrate the data and compute the anomalies

SUNSET has a few modules to perform post-processing on the experimental and the reference datasets. The **Calibration** module performs the bias correction method indicated in the recipe, using the `CSTools::CST_Calibration()` function. 

The **Anomalies** module removes the climatologies using functions like `CSTools::CST_Anomaly()` and `s2dv::Clim()`, and also returns the full fields in case they are needed for any future computations.

```r
# Calibrate the data
data <- Calibration(recipe, data)
# Compute anomalies
data <- Anomalies(recipe, data)
```
**Questions**

(1) Verify that you now have anomaly values instead of the original full field. *Tip*: Use `data_summary()` like in the previous example and pay attention to the new values.

## 5. Evaluate the model skill and compute the probability thresholds

The **Skill** module returns a list of all the evaluation metrics requested in the recipe, in the shape of multi-dimensional arrays. In this case, we will compute three metrics:

- **RPSS (Ranked Probability Skill Score)**: This skill score measures how well a forecast predicts the probability of the tercile categories (below normal, normal and above-normal), compared to the climatology.
- **BSS10 and BSS90 (Brier Skill Score):** This skill score measures how well a forecast predicts the probability of the 10th percentile and 90th percentile extreme events, compared to the climatology.

The `Probabilities()` function returns the probability values for each requested category for the hindcast (and forecast) data, as well as the hindcast percentile values corresponding to each threshold.
```
# Compute skill metrics
skill_metrics <- Skill(recipe, data)
# Compute percentiles and probability bins
probabilities <- Probabilities(recipe, data)
```
**Questions**

(1) What is the structure of `skill_metrics`? Which metrics were computed? What dimensions do they have? *Tip*: use `str()` and `names()`.

(2) What is the structure of `probabilities`? Can you identify the probability categories and the percentiles? *Tip*: use `str()`.

## 6. Plotting the results

Now, let's visualize the information that was computed!

The **Visualization** module will generate the maps we requested in the recipe: Skill maps to visualize the skill distribution of the model, for each metric.

With the significance option in the `Visualization()` function, you can choose whether or not to shade the grid points that are statistically significant in each skill metric plot.

```r
# Plot data
Visualization(recipe, data, 
              skill_metrics = skill_metrics,
              significance = TRUE)
```

Now, you can `cd` to the the output directory and inspect the contents of the `plots/` subdirectory. The plots are PNG files that can be visualized with the `display` command. They have a descriptive name including the content of the plot, the date and the forecast time.
