### Debugging Scorecards module without running the whole recipe

First, you need to run SUNSET asking to produce and save metrics from the Skill module. In the recipe you need to request ´scorecards´ output format and several models and/or initialisation dates and metrics. For instance, in the case of Seasonal climate forecasts, the 12 initialisation dates can be requested. You can also include the scorecard section in the recipe to see which is the default result. 

SUNSET will create an output directory (e.g.: /esarchive/scratch/ptrascas/R/sunset_outputs/recipe_example_scorecards_20240213120100) that will contain a folder called ´output´ in which numeric output files will be save (i.e. NetCDF file). Ensure it contains a folder `Skill` with the results of the verification (e.g. all start dates, metrics and models requested are stored).


To debug it, we will use an interactive session in the WS being in the SUNSET code folder:


```r
source(MODULES)
R
```

We will use the first function [execute_scorecards.R](https://earth.bsc.es/gitlab/es/sunset/-/blob/master/modules/Scorecards/execute_scorecards.R)  on module Scorecard. 


```r
source('tools/libs.R')
source('modules/Scorecards/Scorecards.R')

```

Skip lines 4 to 6. Instead adjuste them to your case:

```
recipe_file <- "/esarchive/scratch/recipe_example_scorecards.yml"
output_dir <- "/esarchive/scratch/ptrascas/R/sunset_outputs/recipe_example_scorecards_20240213120100" # Copy this folder to your directory to use the skill metrics previously calculated
```


Execute lines from 10 to 14:

```r
recipe <- read_yaml(recipe_file)
recipe$Run$output_dir <- output_dir

## Loop over variables
datasets <- recipe$Analysis$Datasets
```

In case we want to adjust some parameters of the final design, we can select the first model and reference by running:

```
system <- 1
reference <- 1
variable <- 1
```
This allows to load and run a small sample to avoid the task taking a long time.

Now we execute the code from "execute_scorecards.R": 

```r
      scorecard_recipe <- recipe 
      scorecard_recipe$Analysis$Datasets$System <- 
        recipe$Analysis$Datasets$System[[system]]
      scorecard_recipe$Analysis$Datasets$Reference <- 
        recipe$Analysis$Datasets$Reference[[reference]]
      scorecard_recipe$Analysis$Variables <-
        recipe$Analysis$Variables[[variable]]
      # Plot Scorecards
      Scorecards(scorecard_recipe)
```


Check the results (by opening the plot). Adjust one parameter, for instance the colorbar size:


```r
scorecard_recipe$Analysis$Workflow$Scorecards$legend_width <- 300 # tip: the best value is 690
```
Re-run the code:
```r
      Scorecards(scorecard_recipe)
```

Check the output again and keep making small changes until you obtain the desired result.

Scorecards code workflow:

1. execute_scorecards.R that loops on variables, system and reference
2. Scorecards() stored under `modules/Scorecards/Scorecards.R` which input is a recipe
3. Then it starts the workflow to create scorecards:
	3.1 LoadMetrics
	3.2 Aggreage metrics
        3.3 Read any information on parameters specified in the recipe (i.e. recipe$Analysis$Workflow$Scorecards$table_label)
        3.4 Create the scorecards

How to know available parameters? SCPlotScorecard.R
https://earth.bsc.es/gitlab/es/sunset/-/wikis/home#scorecards-module


How to use browser()?

The browser() command is very useful for debugging, as it the script stops running as soon as it finds the command in the script. This way you can execute your scripts by parts, making it easier to find and fix a bug.

For example, let's introduce a bug, in this case, a typo in the Scorecards.R script and re run the function:

Open the /modules/Scorecards/Scorecards.R script and in line 40, add an "x" so it reads "namex". 
```r
  var <- recipe$Analysis$Variables$namex
```

Save the file, close it and rerun everything, including loading the Scorecards.R script
(otherwise the typo will not be saved): 

You will encounter this error:

```r
Error in LoadMetrics(system = system, reference = reference, var = var,  : 
  Parameter 'var' must be a character vector with the var names.
```

Now let's go and place the browser() in the Scorecards.R script, for example in line 55.
In the empty line, just write "browser()". Then run everyting again, including loading the modules.

As you can see, the code stoped running in the line you placed the browser(), and in the R console it will appear "Browse[1]"

If you click enter it will execute the next line of code. Keep clicking until it gives you an error, you will see that the error appears in line 79, in the function "LoadMetrics()"
because we introduced a typo. Here it indicates that var should have the names of the variables, but it's reading "namex".

Type Q to stop debugging, go to the Scorecards.R file and fix the bug.





How to use traceback()?


 
 
 

