Earth Sciences

Earth Sciences Wiki

User Tools

Site Tools


tools:rtools

The R tools can be used for postprocessing experiments: loading data, computing prediction scores, indices or calibrate experiments as well as plotting, formating data and saving data. Furthermore, by dividing the data in chunks, you can speed your execution by using startR.

You can join the Earth RTools mailing list to receive the latest updates about news.

The list of functions in each department R package and the R modules with library versions can be found here.

A quarterly meeting is taking place in the department to discuss about the plans and priorities of the R tools (or any other topic we need to discuss). Here you can find a document were the minutes are being gathered:

2021

  • R tools user meeting (Online, 09/07/2021) Slides
  • R tools user meeting (Online, 04/06/2021) Slides
  • R tools user meeting (Online, 07/05/2021) Slides
  • R tools user meeting (Online, 09/04/2021) Slides
  • R tools user meeting (Online, 05/03/2021) Slides
  • R tools user meeting (Online, 05/02/2021) Slides Minutes
  • R tools user meeting (Online, 08/01/2021) Slides Minutes

2020

2019

Please, see specific information for each tool:

Presentations

How to cite

If you are using any of the R packages being developed at the department, you can include a citation in your research items.

  • startR

BSC-CNS and Nicolau Manubens (2021). startR: Automatically Retrieve Multidimensional Distributed Data Sets. R package version 2.1.0. https://earth.bsc.es/gitlab/es/startR/

  • ClimProjDiags

N. Pérez-Zanón, and A. Hunter, 2020: ClimProjDiags: Set of tools to compute various climate indices, version 0.1.0. Barcelona Supercomputing Center, R package, https://CRAN.R-project.org/package=ClimProjDiags

  • CSTools

Núria Pérez-Zanón, Louis-Philippe Caron, Carmen Alvarez-Castro, Lauriane Batte, Jost von Hardenberg, Llorenç LLedó, Nicolau Manubens, Eroteida Sánchez-Garcia, Bert van Schaeybroeck, Verónica Torralba and Deborah Verfaillie (2021). CSTools: Assessing Skill of Climate Forecasts on Seasonal-to-Decadal Timescales. R package version 4.0.0. https://CRAN.R-project.org/package=CSTools

  • s2dverification

Nicolau Manubens, Louis-Philippe Caron, Alasdair Hunter, Omar Bellprat, Eleftheria Exarchou, Neven S. Fučkar, Javier Garcia-Serrano, François Massonnet, Martin Ménégoz, Valentina Sicardi, Lauriane Batté, Chloé Prodhomme, Verónica Torralba, Nicola Cortesi, Oriol Mula-Valls, Kim Serradell, Virginie Guemas, Francisco J. Doblas-Reyes, An R package for climate forecast verification, Environmental Modelling & Software, Volume 103, 2018, Pages 29-42, ISSN 1364-8152,https://doi.org/10.1016/j.envsoft.2018.01.018.

  • s2dv

BSC-CNS, An-Chi Ho and Núria Pérez-Zanón (2020). s2dv: A Set of Common Tools for Seasonal to Decadal Verification. R package version 0.1.0. https://CRAN.R-project.org/package=s2dv

  • Given that these packages are using R environment, you can also include:

R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

Note that the version of R and packages can be updated, you can check the latest reference by open a terminal and running `citation()` for the R version and `citation(“packageName”) for each package. We are also working on a manuscript about CSTools.

Finally, it is recommended to add a sentence in Acknowledgements section of your manuscripts where readers can check the full list of software used. Here is an example from 'How Reliable Are Decadal Climate Predictions of Near-Surface Air Temperature?' (Verfaille et al, 2020):

We acknowledge the use of the s2dverification (Manubens et al. 2018), startR (BSC/CNS and Manubens 2020), SpecsVerification (Siegert 2017), CSTools (Pérez-Zanón et al. 2019), ClimProjDiags (BSC/CNS et al. 2020), and boot (Davison and Hinkley 1997; Canty and Ripley 2020) R (R Core Team 2013) software packages.

R Tips

General R solutions that are useful in the department but doesn't belong exclusively to an in-house R package will be listed here.

1. How to change a CDO version in your open R session

> system('module load CDO/1.5.3-foss-2015a')
The following have been reloaded with a version change:
1) CDO/1.6.3-foss-2015a => CDO/1.5.3-foss-2015a

In this case, CDO version has been changed from a newer to an older version.

Remember that you can see the full list of CDO version by running:

  module av CDO

in your terminal

2. How to load dependencies of R package sf

This package could be used by loading the following modules in this specific order:

  module load R/3.6.1-foss-2015a-bare    
  module load GDAL/2.2.1-foss-2015a-GEOS-3.8.0
  module load PROJ/6.1.1-foss-2015a
  module load GEOS/3.8.0-foss-2015a

Note: Avoid to include them in your bashrc, just load them when it is a requirement. Note2: to use library RNetCDF `module load HDF5/1.10.5-foss-2015a` is required.

3. How to load dependencies of R package rgdal

This package could be used by loading the following modules in this specific order:

- for R 3.2.0:

  module load R/3.2.0-foss-2015a-bare
  module load GDAL/2.1.3-foss-2015a
  module load PROJ/4.8.0-foss-2015a

- for R 3.6.1:

  module load R/3.6.1-foss-2015a-bare
  module load GDAL/2.2.1-foss-2015a-GEOS-3.8.0
  # if necessary add: module load PROJ/5.0.1-foss-2015a

Note: Avoid to include them in your bashrc, just load them when it is a requirement.

4. How to avoid Load error in R 3.6.1: cdo -griddes core dumped

To avoid an error of Load because of the command CDO -griddes, a different version of HDF5 is required:

  module load HDF5/1.8.14-foss-2015a
  

The same requirement also applies to s2dv::CDORemap and startR::CDORemapper.

5. How to use 'rmapshaper' library in Nord3

To correctly use the R library rmapshaper in Nord3, you need to load the following modules in advance:

  module load R protobuf/3.7.1-GCCcore-8.3.0 jq/1.5-GCCcore-8.3.0 nodejs/10.21.0-GCCcore-8.3.0 

6. How to avoid plotting issues (as fuzzy labels) in Nord3

To save good quality plots created in Nord3, the library 'ragg' is necessary. It is already installed, so, in your R code or R session, before generating the plot, load the library, and use agg_png() function to define the name, size and resolution of your plot. Then, create your plot and close the device. Here, there are two examples:

  library(CSTools)
  library(ragg)
  # Ex1:
  agg_png("fig1.png", width = 1000, height = 500, units = 'px',res = 144)
  fcst <- data.frame(fcst1=rnorm(mean=25,sd=3,n=30),fcst2=rnorm(mean=23,sd=4.5,n=30))
  PlotForecastPDF(fcst,tercile.limits=c(20,26))
  dev.off()
  # Ex2:
  agg_png("fig2.png", width = 1000, height = 1000, units = 'px',res = 144)
  PlotMostLikelyQuantileMap(list(a, b, c), lons, lats,
  toptitle = 'Most likely tercile map',
  bar_titles = paste('% of belonging to', c('a', 'b', 'c')),
  brks = 20)
  dev.off()

In case you want to save your plot in .ps format, you don't need this library. You can adapt the following lines to your ploting function:

  postscript(‘name_and_path_to_the_file.ps’)
  anyplotingfunction()
  dev.off()

7. Error * caught segfault * address 0x18, cause 'memory not mapped'

If this error appears, check that the partition `/dev/shm/` is empty. In case, trash files are occupying this partition, the process you are running may fail. Remove the files and re-run your code.

If the error persists, check your code with a smaller data sample to discard a problem with your code since this error message indicates that you are requesting more memory than the available.

tools/rtools.txt · Last modified: 2021/07/09 16:56 by aho