- it is working fine with "ithaca" and "mn"
- use of the script is very simple, fill a few user defined variables and issue the following command from cfu desktops:
 nohup ./data_transfer.sh >& expid.log &

and sea ice integrated diagnostics

No commit message

functions and CFU_load handles all the CMIP5 data stored in /cfunas.
Furthermore, as no definite conclusion has yet been taken about the
assessment of the number of independent data in a time series, I put
back in v20, the CFU_eno function of v18 which relies on the VonStorch
and Zwiers formula and which I bounded the output by the actual number
of data. This avoids your playing with the different versions.
    I also put some default values to a number of parameters in all the
functions to make easier your use of the functions and to avoid any
mistake. If you strictly follows the common diagnostic structure for
using all the functions, send the ensemble means to CFU_corr, CFU_RMS,
CFU_trend and CFU_consist_trend and select only 1 experiment and 1
observational dataset for CFU_ratioRMS, the default parameters should
be correct. Of course, when you know well the function, you can play
with the parameters.

1) CFU_load : The options to loa...

CFU_eno has been modified to apply a more reliable and more stable formula to provide the number of independant data of a time series. The result is still not perfect but it is much better than the previous version which provided crazy results. Your confidence intervals and significance levels for RMSE and correlation should then be modified with the version 19 of the common diagnostics toward more accurate estimations. Furthermore, an option has been introduced in CFU_plotequimap to be able to remove the colorbar. Finally, the ERSST & HadISST dataset are now available also when loading tas in CFU_load (previously, they were only available when loading tos). 

I have prepared a new function called CFU_ratioRMS to compute the ratio of RMSE for two experiments and provide the p-value of the Fisher test with Ho: RMSE1/RMSE2 = 1. Furthermore, a trouble has been detected in the computation of the number of independant data in a time series. The computation of the confidence interval and significance level in CFU_corr and CFU_RMS may have been affected by this trouble as they rely on parametric tests. I advise you to check whether the significance level and confidence intervals of your correlation and RMSE is modified with version 18. Some more details are described below :

1) CFU_ratioRMS : You have to provide three matrix:  two for the 2 experiments you wish to compare and 1 observational dataset. All three input matrix should have the same dimensions. The output matrix contains a dimension of length 2 at the position of the input dimension along which the RMSE has been requested : the ratio of the two RMSE1 + the p-value of the Fisher test.

2) The computation of the number of independant data in a time series rely on a formula provided in the VonStorch & Zwiers and which seems not to be accurate enough. The number of independant data can be larger than the true number of data in the original formulation. I have made a temporary fix which consists in bounding the number of independant data by the true number of data. This may not be an optimal solution but it seems to provide good results on the few examples I have tested. 

No commit message

CFU_load : 'hadcm3dec','miroc4dec','miroc5dec','mri-cgcm3dec','hadcm3his','miroc4his','miroc5his','mri-cgcm3his'

CFU_plotequimap

1) CFU_load : too new arguments leadtimemin and leadtimemax which allows to select a range of leadtimes. 

2) CFU_trend : It now outputs both the trend as before but also the linearly detrended data. So that the output come under the form of a list : $trend, $detrended. By linearly detrended data, I mean that the ensemble-mean trend is removed from each member.

3) CFU_consist_trend : If you provide the observational and model matrix, the trend and detrended data will be computed using only the common period (same idea as in the per-pair computation of anomalies). 

The function works with a different number of members for obs and mod 

It has been included as a new trunk in R, and consits of some post-processing functions coded in R but wihtout the constraint of the 'common_diagnostics.txt' structure. This is useful for 1-D (time series) and 2- or 3-D (lon,lat,time) dataset arrays.   

'HadCRUT3glo'/'HadSST2oc'/'CRUTEM3land'/'NCDCglo'/'NCDCland'/'NCDCoc'/'GISSglo'/'GISland'/'GISSoc'
These datasets are area-averaged time-series : the glo/land/oc part of the arguments reminds of over which surface the tos/tas are averaged (global/ocean/land)
The arguments 'NOAA' and 'UKMO' are now called 'ERSST' and 'HADISST' but 'NOAA' and 'UKMO'  still works for compatibility purposes.
This version is also available as /cfu/pub/scripts/R/common_diagnostics_v15.txt toward which /cfu/pub/scripts/R/common_diagnostics.txt is linked

Added directories for main line of development of CFU postprocessing (trunk/postp) and R diagnostics (trunk/R) scripts.