README.md

Ingredients to make a reproducibility test of EC-Earth 3.1 and later.
Largely based on Paolo Davini's initial scripts to make a climate assessment of EC-Earth

Current limitations and constraints:
* The output must come from EC-Earth 3.1 or later
* The output must be available as MMO, MMA tar files and ICMGG, ICMSH grib files
* The output must consist in 12 month data per file
* The simulation must start with January and finish with December
* The scripts are adapted to compare two and only two experiments

1. Checkout the git project from the BSC archive
------------------------------------------------
```sh
 git clone https://earth.bsc.es/gitlab/es/reproducibility-test.git
 cd reproducibility-test
 git checkout master
```

2. Edit the configuration file
------------------------------
```sh
 vi ./scripts/config.sh
```

3. Pre-process the EC-Earth output
----------------------------------
```sh
cd ./scripts
./preprocess.sh ${exp} ${memb} ${yearb} ${yeare}
```

[ Execute ./preprocess.sh to see the help ]

Process *all* your members of all experiments. This can be done in paralle
by using nohup commands for instance:

```sh
exp=m06e;
for i in 0 1 2 3 4 ;  do 
  sleep 2; 
  nohup ./preprocess.sh ${exp} fc${i} 1850 1869 >& log/log_prpr_${exp}_fc${i} &
done
```


4. Re-grid to the same grid as the observational reference
----------------------------------------------------------

```
./postprocess.sh ${exp} ${memb} ${yearb} ${yeare}
```

5. Compute the Reichler and Kim indices
---------------------------------------
```sh
./scores.sh ${exp} ${memb} ${yearb} ${yeare}
```

NetCDF files and a text file will be available in the directory EC-Earth_diags.
This text file (PI3.txt) consists in scores. By looking at how scores vary from
member to member one can guess if an experiment is reproducible wrt another one.

Once steps 1/ -> 5/ have been done for all members, we can proceed
to ensemble analyses:

6. Ensemble analyses
--------------------
```sh
./ensemble.sh ${exp} ${yearb} ${yeare}
```

Process all your experiments with this script. And then produce the plots comparing the Reichler and Kim indices and the variables differences between two experiments:

7. Run reproducibility diagnostics between two experiments
----------------------------------------------------------
```sh
./compare.sh ${exp1} ${exp2} ${year1} ${year2} ${nmemb}
```

${nmemb} is the number of members that is used to compare the experiments.

This script launch three R scripts:

1. Basic time series (2-m temperature, sea ice extent, precip)
2. The first on make the comparison based on the Reichler-Kim indices with the KS test (script R_scripts/KS_index.R). It produces the plot reichler_kim_scores_stat_${exp1}_${exp2}.ps
3. The second script produces the maps of the differences, with a KS test applied in each grid cell (script map_diff_experiments.R), for the variables 't2m','msl','qnet','tp','ewss','nsss' and 'SICE'. RData files are also saved to redo quickly the maps, changing for example the levels of the colour bars.