tools:s2dverification []

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
==== Objective ====

Set of tools to assess the performance of a model through the computation of typical prediction scores against one or more observational datasets or reanalyses (a reanalysis being a physical extrapolation of observations that relies on the equations from a model, not a pure observational dataset).

==== Description ====

This package is coded in R.

It allows you to load a variable from one or more experiments together with the reference data at the corresponding dates for validation where the reference data can be any observational dataset or reanalysis or several of them. Once  loaded the experimental and reference data are organised into two matrices with similar structures that can be used as input arguments to the other functions to compute anomalies, climatologies, any prediction score and finally plot or animate the results.

If you want to keep abreast of the latest releases, bugs and discussions on s2dverification, subscribe to its mailing list by sending an e-mail to s2dverification-request@bsc.es with the subject 'subscribe'.

s2dverification user manual: {{tools:s2dverification:20160104_nmanubens_s2dverification_2.5.0.pdf|pdf}}

Slides on s2dverification from the tools meeting held on 2014/11/03: {{tools:s2dverification:s2dverification-specs_ga.pdf|SPECS GA s2dverification slides}}

Overview slides and big data issues from BSC-UniCan meeting:
{{tools:s2dverification:20150506_nmanubens_bsc-unican-s2dv-bigdata-overview.pdf|pdf}}{{tools:s2dverification:20150506_nmanubens_bsc-unican-s2dv-bigdata-overview-ppt.ppt|ppt}}

Slides about R, its popularty and tricks for performance, from CES training talks on 2015/05/19: {{tools:s2dverification:20150519_nmanubens_training-talks-R-performance.pdf|pdf}} {{tools:s2dverification:20150519_nmanubens_training-talks-R-performance.ppt|ppt}}

Also you can check the slides used in the s2dverification meetings:

  * 2014/07/11: {{tools:s2dverification:s2dverification_20140711.pdf|pdf}}
  * 2014/11/06: {{tools:s2dverification:s2dverification-20141106.pdf|pdf}}
  * 2014/12/09: {{tools:s2dverification:s2dverification-20141209.pdf|pdf}}
  * 2015/01/20: {{tools:s2dverification:s2dverification-20150120.pdf|pdf}}
  * 2015/02/23: {{tools:s2dverification:s2dverification-20150223.pdf|pdf}}
  * 2015/03/23: {{tools:s2dverification:s2dverification-20150323.pdf|pdf}}
  * 2015/04/30: {{tools:s2dverification:20150430_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20150430_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2015/05/29: {{tools:s2dverification:20150602_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20150602_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2015/07/06: {{tools:s2dverification:20150706_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20150706_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2015/08/06: {{tools:s2dverification:20150806_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20150806_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2015/09/21: {{tools:s2dverification:20150921_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20150921_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2015/10/28: {{tools:s2dverification:20151028_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20151028_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2015/11/30: {{tools:s2dverification:20151130_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20151130_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2016/02/08: {{tools:s2dverification:20160208_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20160206_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2016/03/03: {{tools:s2dverification:20160303_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20160303_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2016/05/04: {{tools:s2dverification:20160504_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20160504_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2016/06/03: {{tools:s2dverification:20160603_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20160603_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2016/09/13: {{tools:s2dverification:20160913_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20160913_nmanubens_s2dverification_update_meeting.ppt|ppt}}
  * 2017/02/20: {{tools:s2dverification:20170220_nmanubens_s2dverification_update_meeting.pdf|pdf}} {{tools:s2dverification:20170220_nmanubens_s2dverification_update_meeting.pptx|pptx}}
  * 2017/09/15: {{tools:s2dverification:20170915_nmanubens_ahunter_s2dverification_update_meeting.pdf|pdf}} {{https://docs.google.com/presentation/d/1N16rWi8jxkaHRu0LbRpK6hRA_Ra9zWzvZPMBHLXMEaM/edit#slide=id.g2549613e5c_0_85|gdoc}}



==== Requirements ====

This package has two system dependencies, CDO and R (>= 2.14.1), apart from the R packages ncdf4, GEOmap, geomapdata, maps, mapproj, abind, parallel and bigmemory but these are installed automatically with <code>install.packages('s2dverification')</code>

==== Repository ====

The link to the GIT repository is [[https://earth.bsc.es/gitlab/es/s2dverification.git|https://earth.bsc.es/gitlab/es/s2dverification.git]]

==== Contact ====

The coordinator of this project is Nicolau Manubens Gil <nicolau.manubens@bsc.es>

Virginie Guemas <virginie.guemas@bsc.es>, Javier Garcia-Serrano <javier.garcia@bsc.es>, Nicolau Manubens Gil <nicolau.manubens@bsc.es>, Ludovic Auger <ludovic.auger@meteo.fr>

As a new user, please register to the mailing list by sending an e-mail with the subject 'subscribe' to the following address, if you are interested in keeping abreast of latest news or internal discussions:
  s2dverification-request@bsc.es
==== How to use ====

To use the R s2dverification functions, open an R session and load the package with the command

<code>library(s2dverification)
</code>

or, if you want to avoid collision of s2dverification function names with function names of other R packages in use in your session, with the command

<code>requireNamespace('s2dverification', quiet = TRUE)
</code>

If you use the second option, you must call the functions in the package as s2dverification::FunctionName() instead of FunctionName()

You can check the list of functions in the package with the command

<code>help(package = s2dverification)
</code>

After the package is loaded, you can check information of a function with the command

<code>?FunctionName
</code>

You'll see a quick description of the aim of the function, the input arguments, the output variables and contact references in case of questions.

To learn how to use these functions you can start by giving a look at the examples in the documentation pages of each function. For more advanced examples of usage you can run the scripts under the directory 'doc' of the s2dverification package ('inst/doc' in the raw GIT repository). Few arguments need to be specified as explained in the headers inside:

<code>  plot_timeseries.R
  plot_maps.R
</code>

=== Using a development version ===

If you want to use an old version or a version still in development of the s2dverification, download the repository folder to your computer and check-out to the branch that contains the version you want to use.

Set the following environment variables with the commands

<code>R_LIBS="/usr/local/lib/R/site-library:/usr/lib/R/library:/usr/lib/R/site-library:/cfu/software/R/debian7"
 export R_LIBS
 _R_CHECK_RD_XREFS_=false
 export _R_CHECK_RD_XREFS_
</code>

Go to the repository directory and build the package tarball with the command

<code>R CMD build /path/to/repository
</code>

this will create a .tar.gz file in the current directory (repository main folder) with the source files compressed, ready to install.

Install the package locally with the command

<code>R CMD INSTALL -l /path/where/to/install /path/to/tarball.tar.gz
</code>

Then you can load the library in R with the command

<code>library(s2dverification, lib.loc = '/path/where/to/install')
</code>

and it will be ready to use.





==== Development ====

=== Package structure ===

In the 's2dverification' project main folder you can see a few files that fulfil the standard R package structure. These files are called the source files.
The structure is as follows:

- Folder 'R': Contains .R files with the code of the functions in the package, one function per file. The name of the file must be the same as the function it contains ending with '.R'.

- Folder 'man': Contains .Rd files with the documentation of the functions and of the package, one file per function and one file for the documentation of the whole package. The name of the file must be the same as the function, ending with '.Rd'. This file should contain a section named 'examples' than contains example code that calls the function being documented.

- Folder 'inst': Contains documentation files, images, etc. that will be provided to the user after the installation of the package. Be careful with the weight of the files, they should be as light as possible.
For example, the folder 'inst/doc' contains demo scripts that show how to use the package. When the user installs the package, they will be able to check the scripts in the folder '/path/to/package/doc'. This folder also contains the folder 'config' which contains two s2dverification configuration files, one used at IC3 and which is loaded by default, and a sample configuration file.

- File 'DESCRIPTION': Contains general information about the package, dependencies, authorship, etc. and is looked up automatically in the package build stage.

- File 'NAMESPACE': Specifies what variable names are used from outside the package and what variable names are provided to the outside.

=== How to develop ===

Download the repository folder to your computer ( git clone [[https://earth.bsc.es/gitlab/es/s2dverification.git|https://earth.bsc.es/gitlab/es/s2dverification.git]] <local_folder> ).

Using GitLab and according to the GIT branching and testing strategy introduced in 2014/11, every time you want to perform a new development, you must follow the steps in the image, explained in detail below.

[[#|{{tools:s2dverification:developing_s2dv.png}}]]

== Developer steps ==

0- Log in GitLab and go to the GitLab project main page

<code>· If a red message apears at the top, you can close it
</code>

1- Open a new issue

<code>· Issues -> + New Issue -> Submit new issue
 · Put title, description and label. Write @username to notify people
 · Click on "Submit new issue"
 · You can then see a list with all the issues. Each is assigned an #issue_number
</code>

2- Create a branch from 'master'

<code>EITHER from GitLab
 · Commits -> Branches -> New branch
 · Put name of the new branch and the name of the branch you check off from, in this case 'master'
 · Click on "Create branch"
 OR from terminal
 · git fetch origin
 · git checkout master
 · git pull
 · git checkout -b branch_name
</code>

3- Perform feature/bugfix commits

<code>· Open terminal
 (· git fetch)(only if created from GitLab)
 · Checkout to the new branch
 · Make changes
 · Last commit must finish with 'Fixes #issue_number'
</code>

4- Check that everything is fine

<code>· Run the script /shared/earth/software/scripts/test-s2dverification branch_name. It will do atomatically:
   a) rebase master into the development branch
   b) build the package tarball
   c) check the tarball
 · If any error raises during rebase do:
   git merge --no-ff master
   fix conflicts
   commit
   try 4- again
 · If any error raises during build or check:
   build and check manually (follow instructions in wiki in section How to develop > How to build and check manually)
   fix errors
   commit
   try 4- again
 · git push origin HEAD
</code>

5- Create a merge request

<code>· Merge Requests -> New Merge Request
 · Choose origin and destiny
 · Click on "Compare branches"
 · Fill in title and description.
   Description field should contain testers referenced with @tester_name
   If any comment is needed put there
   Select a milestone for the feature, so that it's associated to a deadline. If there's no associated milestone you can create one or specify a deadline for the feature to be tested in the text
 · Assign to the project coordinator
 · Click on "Submit merge request"
</code>

== Tester steps ==

6+7-

<code> · Run /shared/earth/software/scripts/test-s2dverification branch_name
 · If you run it from within the s2dverification GIT repository, the clone will be skipped
 · If any error raises please tell the developer
 · An R session will open where you can load the new version of the package with library(name_of_the_package)
 · If you want to test from Rstudio you can load an installation that the test script has generated (see script notifications)
</code>

8- When the tests are performed and passed approve the MR with an approval comment (write :+1: in a message in the MR discussion)

== Coordinator steps ==

9- Validate and perform the merge clicking on "Accept Merge Request", with "Remove source-branch" marked.

<code>· It is important before accepting the merge request to run the test script on the branch which is going to be merged.
 · Even at that stage conflicts could appear. In that case, solve them meeting the developers involved.
 · After merging and removing a branch from GitLab, carefully delete the branch from your local Git repository with git branch -D branch_name
</code>

=== Steps to build and check a package manually ===

Set the following environment variables with the commands

<code>_R_CHECK_RD_XREFS_=false
 export _R_CHECK_RD_XREFS_
</code>

You can place the commands above in your .bashrc file if you develop in this package frequently.

Go to the repository directory and build the package tarball with the command

<code>R CMD build --resave-data /path/to/repository
</code>

this will create a .tar.gz file in the current directory (repository main folder) with the source files compressed, ready to install.

Check that the tarball is installable with the command

<code>R CMD check --no-examples /path/to/tarball.tar.gz
</code>

This will advice you if there is any problem with the added changes and will run the example code in the '.Rd' files.
Also will generate in your current directory the folder 's2dverification.Rcheck', that contains the log files of a set of tests to ensure the tarball will be installable.
This folder also contains the folder 's2dverification' with an installation of the package and the file 's2dverification-manual.pdf', a handsome user manual generated automatically from the files in the folder 'man'. A copy of it is being kept in the project main folder.

If you wish, for tidy purposes, you can run the two previous commands in the project main folder. The tarball and the .Rcheck folder will be generated there breaking the package structure, but will be ignored when you push the changes to the repository.

If you wish, you can enable the execution of the example code removing the parameter '–no-examples':

<code>R CMD check /path/to/tarball.tar.gz
</code>

The example execution takes a long time, but it is run at least before every release to CRAN.

Optionally you can start R and load the installation of the package in the 's2dverification.Rcheck' repository with the command

<code>library(s2dverification, lib.loc = '/path/to/s2dverification.Rcheck/')
</code>

and check that the modified functionalities work properly again.

=== Steps to add a new function to a package ===

  * Put the code of the function in a file named 'FunctionName.R' and place it in the 'R' folder.
  * Make sure the code of the function follows the style rules (information below header, history, …).
  * Load the function with source() in an R session.
  * Type prompt(FunctionName). This will generate an .Rd file in the working directory.
  * Place the .Rd file into 'man' folder.
  * Fill in the .Rd file with the function documentation. Most of the text in this file should coincide with the text in the comments below the function header.

=== Reminder ===

Each function in the folder 'R' has a corresponding .Rd file in the 'man' folder, and the text in the .Rd will be used to generate the user manual and will appear when you type ?FunctionName.

=== New Functionalities needed by the CFUers ===

|New function or Improvement of an existing function |Who needs it ? |Who volunteers to do it ? |Planned delivery date  |
|Plot of arrows (winds or currents) superimposed on a field in PlotEquiMap |Chloe, Eleftheria |Chloé |Done  |
|Plot of arrows (winds or currents) superimposed on a field in PlotStereoMap |Neven |  |  |
|Bootstrap as additional significance tests in all the score functions |all |LP |  |
|Probabilistic scores |all |Nicolau, Veronica, Chloe, Omar, Virginie |To be decided once the strategy is settled  |
|Statistical significance of difference in spread |all |  |  |
|Kendall's tau (rank correlation) |LP  |LP |Done  |
|Function to perform Poisson regression |LP  |LP |  |
|Plot an ORCA field directly on the ORCA grid |Neven, Eleftheria, Francois |  |  |
|Function to test the difference of two correlation |all |Omar? |  |

==== Maintenance ====

=== Branch naming conventions ===

Main branch: master

Development branches: develop-feature or feature

Bugfix branches: develop-bugfixes-2.3.1 (when last release was 2.3.1)

Docfix branches: develop-docfixes-2.3.1 (when last release was 2.3.1)

Hotfix branches: develop-hotfixes-2.3.1-a (when last release was 2.3.1)

Upgrade branches: develop-upgrade-to-2.3.2 (when last release was 2.3.1)

Version numbers will be bumped as described in the Semantic Versioning conventions. See http://semver.org.

=== Steps to release ===

1- Check all open issues to see if some can be solved easily before releasing.

2- Open MR for all open bugfix branches.

3- Open MR for all open docfix branches.

4- Open MR for all open hotfix branches.

5- Close all MRs possible. For each:

<code>· run test script before accepting
 · ls -la everywhere to check no trash/huge files leak into master. Update .gitignore and .Rbuildignore if needed
 · accept (removing source branch)
 · git checkout master
 · git fetch origin
 · git pull
</code>

6- Run test script. Then run manually R CMD check without parameter –-no-examples and with parameter -–as-cran if releasing on CRAN. Try to include in test scripts all bugs detected since last release.

7- If small errors appear, fix in for example develop-hotfixes-2.3.1-b and go to 3-.

8- Open upgrade branch and open MR and merge as in 4-. Update at least:

<code>· DESCRIPTION (date, version, dew depends, new authors)
 · NAMESPACE (new depends)
 · man/package-s2dverification.Rd
 · pdf
</code>

9- Submit on CRAN.

10- Create a tag (if from GitLAB, do a 'git fetch origin' after).

11- Add any special messages in R code or add log-registering commands.

12- Build and install in /cfu/software, and in amdahl and in moore for each module possible.

13- Send release e-mail to s2dverification list, with all news since previous release.

14- Update change log on wiki page.

==== Style Guide ====

You can check the style guide for the s2dverification package [[:tools:style_guides:r|here]]

==== Change log ====

^Version name ^Date ^New features  |
|CFU_tools |????-??-?? |  |
|s2dverification 2.0 |2013-08-02| · Use of the standard R package structure.\\ · Use of the google's R style guide.\\ · Functions that involved RClim set of funcions have been kept apartfrom the package (AnimVsLTime, BlueRed, PlotMap, ProjMap) as well as the authors.\\ · New functions have been added: Alpha, EnoNew, Filter, FitAcfCoef, FitAutocor,  GenSeries,  Spectrum.\\ · Extended help.  |
|*s2dverification 2.1 |2014-01-23 | · Demo scripts 'plot_timeseries.R' and 'plot_maps.R' available in the 'inst/doc' directory in the s2dverification repository.\\ · Documentation on how to specify the grids and masks to the function Load() has been added to its help page, code and package manual.  |
|*s2dverification 2.1.1 |2014-06-27 | · Important bugfixes in Clim(), and other bugfixes in Season(), Smoothing() and Plot2VarsVsLTime().\\ · Bugs with the new version of CDO have been fixed.\\ · Package is under licence GPLv3.\\ · Example datasets and code for each function have been added.  |
|s2dverification 2.1.2 |2014-07-09 | · Bugfix in CDO calls. |
|s2dverification 2.2.0 |2014-12-16 | · ACC provides confidence intervals obtained with bootstrap method\\ · Function to plot ACC score\\ · Function to plot variables on a polar stereographic projection\\ · Possibility of loading observations only\\ · Possibility to load more ice variables\\ · Adjustable significance level in the Corr function\\ · Adjustable number size in ColorBar |
|s2dverification 2.3.0 |2015-03-02 | · Configuration file mechanism to specify new dataset or variable paths, grids, etc.\\ · New parameters in Load() to specify maximum and minimum values.\\ · New supported dataset formats. See '?Load' in R after loading s2dverification for more information.\\ · More efficient memory usage in Load() and usage of multiple parallel processes (faster).\\ · NetCDF4 + OPeNDAP support |
|s2dverification 2.3.1 |2015-03-09 | · Loading observations only fixed\\ · Loading only one leadtime fixed\\ · Loading a 2D variable when the first observation was not stored in file-per-dataset format fixed\\ · Parameter 'ncores' changed to 'nprocs'\\ · Improvements in configuration file mechanism and documentation |
|s2dverification 2.3.2 |2015-04-23 | · New CRPS() function to compute the continuous ranked probability score for ensemble forecasts.\\ · New ProbBins() function to compute probabilistic information of a forecast relative to a threshold or a quantile.\\ · Load() stops and warns if the masks provided are not in the correct grid.\\ · Load() didn't apply, as expected, the same masks in observations as in experiments when possible. Now fixed.\\ · Enhancement in Clim() documentation.\\ · Enhancements in Load() and configuration file documentation.\\ · HadSLP dataset is now loadable  |
|*s2dverification 2.4.0 |2015-07-27 | · Option to draw rectangles in PlotEquiMap()\\ · Motification of Corr() to accomodate ranked correlation\\ · Added the possibility to load the second set of HadCM3 decadal data (i3p1)\\ · Added functions to assist in manipulating the configuration file\\ · Improved examples that use extremely reduced experimental and observational datasets\\ · Uniformized documentation style\\ · Added possibility to configure dimension names to look for inside NetCDF files\\ · Added the possibility to load ESA observations from SMHI\\ · Fixed bug that happened in some cases when a common grid is not specified | 
|s2dverification 2.4.1 |2015-07-31 | · Bug that didn't allow to load experimental datasets with the month or day in the file paths now fixed |
|s2dverification 2.4.2 |2015-08-13 | · Added 'methods' package as dependency so that Load() doesn't crash when run from Rscript |
|s2dverification 2.4.3 |2015-09-03 | · Updated package title and description\\ · Fixed bug when loading observational global means\\ · NMME models now loadable | 
|s2dverification 2.4.4 |2015-10-01 | · Fixed bug when detecting NetCDF dimension names\\ · Updated configuration file to BSC repository\\ · eucleia observations now loadable | 
|s2dverification 2.4.5 |2015-10-19 | · Fixed bugs in plot_timeseries.R sample script\\ · Fixed wrong entries in BSC configuration file for some ice variables | 
|s2dverification 2.4.6 |2015-10-22 | · Updated plot_timeseries.R to new paths and to 'ncdf4' | 
|s2dverification 2.4.7 |2015-11-15 | · Updated plot_timeseries.R to new paths and to 'ncdf4'\\ · Improved performance when retrieving subsets of data (regions of earth or time periods)\\ · Added possibility to use Load() without a configuration file. See details on parameters 'exp' and 'obs' in ?Load\\ · Load() now returns plenty of metadata. Highlighted:\\ · Paths to all loaded files\\ · Paths to not found files\\ · Stamp with all the provided parameters to exactly reproduce any Load() call\\ · The name of the common grid (if any), following CDO's naming conventions\\ · Other enhancements in Load():\\ · Enhanced error handling and error messages\\ · Added "progress bar"\\ · Detects automatically grid of the files. No need to specify it\\ · Detects automatically if the requested variable is 2-dimensional or global mean. No need to specify it\\ · Possibility to load observations only, from a limited period of time only\\ · Possibility to load NetCDF files with disordered dimensions\\ · Removed system dependency of 's2dverification' to NCO and some GNU tools\\ · Simplified configuration file: removed lists of variables and reduced from 5 tables to 2, one for experimental datasets and one for observational datasets. You can convert old configuration files to the new format with the script in /shared/earth/software/scripts/convertConfig.R as follows:\\   /shared/earth/software/scripts/convertConfig.R /path/to/configfile.conf\\ · Fixed and updated the sample script plot_timeseries.R\\ · Fixed wrong entries in BSC configuration file for some ice variables|  
|s2dverification 2.4.8 |2015-11-19 | · List of loaded and not-loaded paths now displayed correctly | 
|s2dverification 2.4.9 |2015-11-19 | · Corrected bug when loading longitudes across the greenwich | 
|s2dverification 2.4.10 |2015-11-30 | · Corrected bugs when loading longitudes across the greenwich | 
|s2dverification 2.4.11 |2015-12-07 | · Load() can now load files with multiple grids defined inside | 
|*s2dverification 2.5.0 |2016-01-04 | · Fixed bugs when using masks in Load()\\ · Can now specify masks with paths to NetCDF files | 
|s2dverification 2.6.0 |2016-06-06 | · Updated configuration file.\\ · Functions to compute variability modes and project data on them: EOF() and ProjectField(), by Fabian, Virginie, Lauriane, Martin.\\ · Function to compute co·variability modes: SVD(), by Javi.\\ · Function to compute the NAO: NAO(), by Fabian, Virginie, Lauriane, Martin.\\ · Brier score/skill score accounting for small ensemble/start date sample size: UltimateBrier(), by Nico, Omar, Verónica, Chloé and Virginie.\\ · K-means spatial clustering: Cluster(), by Neven.\\ · Synthetic data generator: ToyModel(), by Omar.\\ · Tropical cyclone downscaling: StatSeasAtlHurr(), by Luis·Philippe.\\ · Function to composite fields: Composite(), by Neven.\\ · Function to generate map animations: AnimateMap(), by Virginie.\\ · Function to plot time series with box-and-whisker plots: PlotBoxWhisker(), by Fabian, Lauriane and Nico.\\ · Now possible to disable computation of confidence intervals or p-values in ACC(), Corr(), RatioRMS(), RatioSDRMS(), RMS(), RMSSS(), Spread() and Trend().\\ · Now possible to adjust confidence level in all functions that provide confidence intervals: ACC(), Corr(), RMS(), Spread() and Trend().\\ · Now possible to plot arrows in PlotEquiMap(), by Chloé.\\ · Now possible to save plots in multiple formats, to file or onscreen from all plot functions.\\ · Objects returned by Load() are now closer to the format in downscaleR. The initial and end date of each time step is provided now.\\ · Enhancements in Smoothing().\\ · Load() now stops if the tag $START_DATE$/$YEAR$+$MONTH$+$DAY$ is not in the path pattern of any of the experiments. | 
|s2dverification 2.6.1 | 2016-06-22 | · Fixed bug in Load() when loading more than 1 experimental data set or more than 1 observational data set. |  
|s2dverification 2.6.2 | 2016-06-27 | · Fixed bug in Load() when interpreting some path patterns. | 
|s2dverification 2.6.3 | 2016-06-28 | · Fixed bug in Load() when interpreting some path patterns.\\ · Fixed wrong entries in configuration file for some ice variables for 'nsidc-nasa' and 'hadisst_v1.1'. | 
|s2dverification 2.6.4 | 2016-07-01 | · Added functions for automatizing multi-pannel map plots: PlotMultiMap() and GenBrksCols(). | 
|s2dverification 2.6.5 | 2016-07-13 | · Added support in Load() for NetCDF files with 'add_offset' and 'scale_factor'.\\ · Updated default BSC configuration file with missing reconstruction data sets. | 
|s2dverification 2.7.0 | 2016-08-24 | · Enhanced PlotEquiMap() and PlotStereoMap() with lots of new options and fixed issues:\\    - Colour bar with triangle ends and lots of new features.\\    - Margins, labels, ticks, colour bar, titles now arranged properly.\\    - Now possibile to adjust colour and border of continents, size, colour and type of contour lines, size of labels, ticks and margins, colour and width of boxes, etc.\\    - Draw multiple superimposed dot/symbol layers.\\    - Draw boxes in PlotStereoMap().\\    - PlotStereoMap() with bounding circle.\\ · Added function PlotLayout() to automatically generate complex figure layouts and populate with plots from multi-dimensional arrays.\\ · Fixed and updated corrupted example scripts (required for new auto-ecearth releases to work).\\ · Added function Subset() to easily take chunks of data arrays.\\ · Fixed bug in Load() under some particular configurations.\\ · Enhanced margins in PlotAno().\\ · Updated sample data to be together with metadata as provided by Load().\\ · Updates and fixes in the BSC Load() configuration file. | 
|s2dverification 2.7.1 | 2016-09-07 | · Few enhancements in colour bar and map plot functions. | 
|s2dverification 2.7.2 | 2016-09-29 | · Enhancements to adjust size and resolution of plots.\\ · Enhancements in Load() to support some specific path patterns. Now possible to send additional variables to the configuration file, specific of each dataset in 'exp' or 'obs'.| 
|s2dverification 2.7.3 | 2016-11-09 | · Bugfixes in plot_timeseries.R example script.\\ · Small bugfix in Load().| 
|*s2dverification 2.8.0 | 2017-02-13 | · Changed licence from GPL-3 to LGPL-3.\\ · New veriApply compatible score functions (.BrierScore, .Corr, .RMS, .RMSSS, .RatioRMS, .RatioSDRMS and .Trend).\\ · New function CDORemap, to interpolate R arrays with CDO.\\ · New function ArrayToNetCDF, to save R arrays with metadata in NetCDF files.\\ · Enhanced plot_timeseries.R and plot_maps.R example scripts to work with file-per-chunk data, for auto-ecearth v3.2.1a.\\ · Added colour-blind colour bars for the map plots.\\ · Added warning in Load when extrapolating data.\\ · Bugfix in ProbBins when called with cross-validation.\\ · Enhanced documentation.\\ · Adapted UltimateBrier to SpecsVerification 0.5-0.\\ · Enhancements to adjust size and resolution in plotting functions.\\ · Solved PlotEquiMap bug when values equal to the lower limit.\\ · Bugfix in Ano.\\ · Bugfix in PlotVsLTime.\\ · Small update in the configuration file.|
|s2dverification 2.8.1 | 2017-02-15 | · Bugfix in ColorBar. |

*This release is also available on CRAN

==== Presentation ====

This presentation given at the SPECS general assembly 2014, summarize the organization of the package, the main functions, some examples and the main lines of development.
{{tools:s2dverification:presentation.pdf|pdf}}

==== Tutorials ====

The following tutorial given at the SPECS general assembly 2014 shows the basic feature of s2dverification. All the examples are done with the data included in the package on CRAN, so it can be done outside of IC3. All the example are corrected.

Time estimation: 1hour.

{{tools:s2dverification:tutorial_specs.pdf|pdf}}

The second tutorial is an extract of the tutorial done to learn how to use s2dverification for the ICTP-WRCP extreme event summer school. It is longer and more complete than the first one, and it uses the ENSEMBLES seasonal forecast, hence it cannot be done outside of the IC3.

Time estimation: 3hours.

{{tools:s2dverification:tutorial_ictp.pdf|pdf}}

==== Known related tools ====
=== Not programmed in R ===
Statistics + Verification:
^ Name ^ Last known release date ^ Developed at ^ Platform ^ Features ^
| MET (Model Evaluation Tools) | 2015-10 | NCAR-DTC | Linux | · Mature package\\ · Data reformatting, aggregation, regridding and masking\\ · Statistics\\ · Probabilistic scores\\ · Visualization\\ · Tropical cyclone analysis |
| EVS (Ensemble Verification System) | 2015-03 | NOAA | Java* | · Hydrologic and hydrometeorological variable analysis\\ · Data pairing, aggregation\\ · Statistics\\ · Deterministic and probabilistic scores with confidence intervals and bootstrapping\\ · Visualization tools (some in R) |
| Climate Explorer | 2016-01 | KNMI | Web browser | · Some available datasets\\ · One can upload data after logging in\\ · Seasonal forecast verification |
| CPT (Climate Predictability Tool) | 2016-01 | IRI | Windows | · Seasonal climate verification\\ · Input: ASCII\\ · Outputs: Maps and series in ASCII or jpg\\ · Scores: PCR, CCA, MLR, GCM\\ · Bootstrapping |
| verif | 2015-12 | UBC | Python* | · Verification of weather forecasts\\ · Reads tabular data\\ · Deterministic and probabilistic scores\\ · Plots data and scores |
| MeteoLab | 2016-02 | UC-CSIC | Matlab* | · Visualisation\\ · Regression\\ · EOF and clustering\\ · Deterministic and probabilistic verification\\ · Statistical downscaling\\ · Neural + bayesian networks |
| CLIK (Climate Information Toolkit) | 2014-10 | APCC | Web browser | · Download and request of model data\\ · Provides verification\\ · Downscaling\\ · Visualisation |
| ESMValTool | 2015-12 | SMHI, Leeds, MetOffice, ... (CMIP) | Python* | · CMORization\\ · Workflow manager\\ · Revised repository of verification scripts in any language\\ · Data retrieval tools if using python or NCL |
| VACUMM | -- | -- | Python* | · Validation, diagnostics of ocean multi model data\\ · http://www.ifremer.fr/vacumm/contents.html |
*: Supported on any platform that has an interpreter of the specified programming language

Visualisation:
^ Name ^ Last known release date ^ Developed at ^ Platform ^ Features ^
| Ferret |||||
| GrADS |||||
| Sosie |||||
| Panoply |||||

Programmed in R:

  * downscaleR
  * loadeR
  * visualizeR. Examples of the available plotting functions in visualizeR can be found {{tools:s2dverification:20160927_visualizeR_plots.pdf|here}}.
  * SpecsVerification. Examples of the available plotting functions in SpecsVerification can be found {{tools:s2dverification:20160915_SpecsVerification_plots.pdf|here}}. 
  * easyVerification
  * esd
  * wux
  * rnoaa
  * RCMIP
  * ocean
  * RNCEP
  * cruts
  * GhcnDaily
  * RghcnV3
  * cmsaf
  * meteoForecast
  * clim.pact
  * climdex.pcic - Pacific Climate Impacts Consortiums's (PCIC) implementation (in R) of the Climdex routines for calculating the ETCCDI's 27 climate extreme indices.
User Tools

Site Tools

Page Tools
