Ingredients to make a reproducibility test of EC-Earth 3.1 and later.
Largely based on Paolo Davini's initial scripts to make a climate assessment of EC-Earth

Current limitations and constraints:
* The output must come from EC-Earth 3.1 or later
* The output must be available as MMO, MMA tar files and ICMGG, ICMSH grib files
* The output must consist in 12 month data per file
* The simulation must start with January and finish with December
* The scripts are adapted to compare two and only two experiments

1/ Checkout the git project from the BSC archive
------------------------------------------------
>> git clone https://earth.bsc.es/gitlab/fmassonnet/reproducibility.git
>> git checkout master

2/ Edit the configuration file
------------------------------
>> vi ./scripts/config.sh

3/ Pre-process the EC-Earth output
----------------------------------
>> cd ./scripts
>> ./preprocess.sh ${exp} ${memb} ${yearb} ${yeare}

[ Execute ./preprocess.sh to see the help ]

Process *all* your members of all experiments. This can be done in paralle
by using nohup commands for instance:

>> exp=m06e; for i in 0 1 2 3 4 ; do sleep 2; nohup ./preprocess.sh ${exp} fc${i} 1850 1869 >& log/log_prpr_${exp}_fc${i} &    done

4/ Re-grid to the same grid as the observational reference
----------------------------------------------------------
>> ./postprocess.sh ${exp} ${memb} ${yearb} ${yeare}

5/ Compute the Reichler and Kim indices
---------------------------------------
./scores.sh ${exp} ${memb} ${yearb} ${yeare}

NetCDF files and a text file will be available in the directory EC-Earth_diags.
This text file (PI3.txt) consists in scores. By looking at how scores vary from
member to member one can guess if an experiment is reproducible wrt another one.

Once steps 1/ -> 5/ have been done for all members, we can proceed
to ensemble analyses:

6/ Ensemble analyses
--------------------
./ensemble.sh ${exp} ${yearb} ${yeare}

Process all your experiments with this script. And then produce the plots comparing the Reichler and Kim indices and the variables differences between two experiments:

7/ Run reproducibility diagnostics between two experiments
----------------------------------------------------------
./compare.sh ${exp1} ${exp2} ${year1} ${year2} ${nmemb}

${nmemb} is the number of members that is used to compare the experiments.

This script launch three R scripts:

1/ Basic time series (2-m temperature, sea ice extent, precip)

2/ The first on make the comparison based on the Reichler-Kim indices with the KS test (script R_scripts/KS_index.R). It produces the plot reichler_kim_scores_stat_${exp1}_${exp2}.ps

3/ The second script produces the maps of the differences, with a KS test applied in each grid cell (script map_diff_experiments.R), for the variables 't2m','msl','qnet','tp','ewss','nsss' and 'SICE'. RData files are also saved to redo quickly the maps, changing for example the levels of the colour bars.