README.md

Ingredients to make a reproducibility test of EC-Earth 3.1 and later.
Largely based on Paolo Davini's initial scripts to make a climate assessment of EC-Earth

Current limitations and constraints:
* The output must come from EC-Earth 3.1 or later
* The output must be available as MMO, MMA tar files and ICMGG, ICMSH grib files
* The output must consist in 12 month data per file
* The simulation must start with January and finish with December
* The scripts are adapted to compare two and only two experiments

# Checkout the git project from the BSC archive

```sh
 git clone https://earth.bsc.es/gitlab/es/reproducibility-test.git
 cd reproducibility-test
 git checkout master
```

# Edit the configuration file

```sh
 vi ./scripts/config.sh
```

# Copy Climate_netcdf  

```sh
cd scripts
cp /esarchive/releases/models/ecearth/Climate_netcdf
```

# Pre-process the EC-Earth output

```sh
cd ./scripts
./preprocess.sh ${exp} ${member} ${yearb} ${yeare}
```

[ Execute ./preprocess.sh to see the help ]

Process *all* your members of all experiments. This can be done in paralle
by using nohup commands for instance:

```sh
exp=m06e;
for i in 0 1 2 3 4 ;  do 
  sleep 2; 
  ./preprocess.sh ${exp} fc${i} 1850 1869 >& log/log_prpr_${exp}_fc${i} 
done
```


# Re-grid to the same grid as the observational reference


```
for member in 0 1 2 3 4 ;  do
  sleep 2;
  nohup ./postprocess.sh ${exp} fc0${member} ${yearb} ${yeare}
done

```

# Compute the Reichler and Kim indices

```sh
./scores.sh ${exp} ${member} ${yearb} ${yeare}
```

NetCDF files and a text file will be available in the directory EC-Earth_diags.
This text file (PI3.txt) consists in scores. By looking at how scores vary from
member to member one can guess if an experiment is reproducible wrt another one.

Once steps 1/ -> 5/ have been done for all members, we can proceed
to ensemble analyses:

# Ensemble analyses

```sh
./ensemble.sh ${exp} ${yearb} ${yeare}
```

Process all your experiments with this script. And then produce the plots comparing the Reichler and Kim indices and the variables differences between two experiments:

# Run reproducibility diagnostics between two experiments

```sh
./compare.sh ${exp1} ${exp2} ${year1} ${year2} ${number_of_members}
```

This script launch three R scripts:

1. Basic time series (2-m temperature, sea ice extent, precip)
2. The first on make the comparison based on the Reichler-Kim indices with the KS test (script R_scripts/KS_index.R). It produces the plot reichler_kim_scores_stat_${exp1}_${exp2}.ps
3. The second script produces the maps of the differences, with a KS test applied in each grid cell (script map_diff_experiments.R), for the variables 't2m','msl','qnet','tp','ewss','nsss' and 'SICE'. RData files are also saved to redo quickly the maps, changing for example the levels of the colour bars.