Differences

This shows you the differences between two versions of the page.

--- tools:rtools [2023/10/31 10:06]
erifarov [Presentations]
+++ tools:rtools [2024/05/02 14:43] (current)
vagudets [R Tips]
@@ Line 1: / Line 1: @@
-The R tools can be used for postprocessing experiments: loading data, computing prediction scores, indices or calibrate experiments as well as plotting, formating data and saving data. Furthermore, by dividing the data in chunks, you can speed your execution by using startR.
+At BSC-ES there is a collaborative effort to share resources for data analysis in R and develop some in-houseR tools that are maintained by the R developer team. The R tools can be used for postprocessing experiments: loading data, computing prediction scores, indices or calibrate experiments as well as plotting, formating data and saving data. Furthermore, by dividing the data in chunks, you can speed your execution by using startR. You will find more information about our in-house packages further down, in the 'In-house Packages' section.
 You can join the [[http://mailman3.bsc.es/postorius/lists/earth-rtools.bsc.es/|Earth RTools mailing list]] to receive the latest updates about news. Check the list of R tips below to learn about **BSC-ES infrastructure** and other R tips.
-The list of functions in each department R package and the R modules with library versions can be found [[https://docs.google.com/spreadsheets/d/1gA-O1WkgH50VnyxOz9gbec_DsyvWMkyF-9SlFANXLqI/edit#gid=1328408011| here]].
+===== Getting started: using R at BSC-ES =====
-A quarterly meeting is taking place in the department to discuss about the plans and priorities of the R tools (or any other topic we need to discuss). Here you can find a document were the minutes are being gathered:
+==== Environment Modules ====
-  * [[https://docs.google.com/document/d/1VqnwgeimqNLODfqAyPjIY9XprGD5JQ-NPqzQF6nNDS0/edit?usp=sharing&s|Minutes until June 2022]]
-  * [[https://docs.google.com/document/d/1mdLCS7SC5-8NwAB_gAzUsEnJN4yYEYmAlVys0qT8Edk/edit?usp=sharing|Minutes since July 2022]]
+The Workstations, the BSC-ES Hub and the HPC machines use [[https://earth.bsc.es/wiki/doku.php?id=library:environment_modules | environment modules]], which are maintained by our IT team. Each module contains a set of software packages that can only be used if the module has been loaded in the environment.
+For example, to load CDO and R in the workstation, you should load the corresponding modules with the 'module load' command:
+    module load CDO/1.9.8-foss-2015a
+    module load R/4.1.2-foss-2015a-bare
+The currently maintained R module versions are:
+  * Workstations: R/4.1.2-foss-2015a-bare (See: [[https://earth.bsc.es/wiki/doku.php?id=computing:workstations| Computing - Workstations]])
+  * BSC-ES Hub: R/4.2.1-foss-2021b (See: [[https://earth.bsc.es/wiki/doku.php?id=computing:bsceshub|Computing - Bsceshub (Testing phase)]])
+  * Nord3v2: R/4.1.2-foss-2019b (See: [[https://earth.bsc.es/wiki/doku.php?id=computing:nord3-v2|Computing - Nord3v2]])
+  * CTE-AMD: R/4.1.2-foss-2019b (See: [[https://earth.bsc.es/wiki/doku.php?id=computing:amd|Computing - AMD]])
+Check the wiki page for each machine to see if you need to follow any additional steps to be able to load the correct modules.
+==== R packages ====
+The R modules contain the latest released version of our in-house R packages, as well as many other R packages that may be used by people in the department.
+There is no need to install any packages locally by yourself; you can load the corresponding module and check if the R package is already installed. If it is not installed, you may open an issue in the Requests GitLab project (https://earth.bsc.es/gitlab/es/requests/-/issues) and tag Stamen Miroslavov Minkov (@smirosla) to ask him to install it in the machines.
+Some packages may require additional modules; see the ‘R tips’ section below.
+The list of R packages installed in the latest maintained R modules (as of 2024-02-02) can be found {{ :tools:r_modules_and_packages_20240202.ods |here}}, and a list of functions in each department R package (as of 2021-01-18) can be found [[https://docs.google.com/spreadsheets/d/1gA-O1WkgH50VnyxOz9gbec_DsyvWMkyF-9SlFANXLqI/edit#gid=1328408011| here]] or in the documentation for each package on CRAN.
+==== Where to run R ====
+  * To open an R session in the terminal, simply type ‘R’ after loading the R module.
+  * RStudio Server is available on the Workstations and Nord3v2. See the section ‘RStudio Server’ below. RStudio Server is not yet available on the Hub, but you can use VSCode. See the ‘R tips’ section below.
+  * To use Jupyter Notebooks in the BSC-ES Hub, see the [[https://earth.bsc.es/wiki/doku.php?id=computing:howto:jupyter_notebooks#r_experimental | Jupyter Notebook wiki page]]. For the workstations, the up-to-date R version is not working with Jupyter Notebooks.
 ===== R User Meeting =====
+A monthly meeting takes place in the department to discuss about the plans and priorities of the R tools (or any other topic we need to discuss). Here you can find a document were the minutes are being gathered:
+  * [[https://docs.google.com/document/d/1VqnwgeimqNLODfqAyPjIY9XprGD5JQ-NPqzQF6nNDS0/edit?usp=sharing&s|Minutes until June 2022]]
+  * [[https://docs.google.com/document/d/1mdLCS7SC5-8NwAB_gAzUsEnJN4yYEYmAlVys0qT8Edk/edit?usp=sharing|Minutes since July 2022]]
+====2024====
+  * R tools user meeting (Hybrid Mode, 01/02/2024){{ :tools:r_user_meeting_20240201.pdf | Slides}}
+  * R tools user meeting (Hybrid Mode, 04/04/2024){{ :tools:r_user_meeting_20240404.pdf | Slides}}
+  * R tools user meeting (Hybrid Mode, 02/05/2024){{ :tools:r_user_meeting_20240502.pdf | Slides}}
 ====2023====
+  * R tools user meeting (Mix Mode, 14/12/2023){{ :tools:r_user_meeting_20231214.pdf | Slides}}
+  * R tools user meeting (Mix Mode, 02/11/2023){{ :tools:r_user_meeting_20231102.pdf | Slides}}
   * R tools user meeting (Mix Mode, 05/10/2023){{ :tools:r_user_meeting_20231005.pdf | Slides}}
   * R tools user meeting (Mix Mode, 07/09/2023){{ :tools:r_user_meeting_20230907.pdf | Slides}}
@@ Line 54: / Line 92: @@
   * R tools user meeting (BSC, 07/11/2019) {{ :tools:rtools_user_meeting_20191107.pdf |Slides}}{{ :tools:rtools_user_meeting_minutes_20191107.pdf |minutes}}
-===== Packages =====
+===== In-house Packages =====
 Brief Summary and links to in-house packages: {{ :tools:package_summary.pdf | Package Summary}}
@@ Line 70: / Line 108: @@
 ===== Presentations =====
+  * {{ :tools:r_tool_roadmap_2024.pdf |R Tools Roadmap 2024}}
   * {{ :tools:day2_04_handson_r_tools.pdf | BSC Training (PATC) 2023 - Climate Forecast Analysis hands-on tutorial: R tools}}
   * {{ :tools:create_r_package_in_bsc-es.pdf | Create R Package in BSC-ES}} (2023-05-26)
@@ Line 109: / Line 148: @@
 If the repository is **not public**, see {{ :tools:source_functions_from_non-public_gitlab_repo.pdf | this slide}}
-===== How to cite =====
+===== How to cite R Tools =====
-If you are using any of the R packages being developed at the department, you can include a citation in your research items.
+If you use any R packages, both developed in or outside of the department, you can include a citation in your research items.
-  * **startR**
+//Reference//: https://ropensci.org/blog/2021/11/16/how-to-cite-r-and-r-packages/
-  BSC-CNS and Nicolau Manubens (2021). startR: Automatically Retrieve Multidimensional Distributed Data Sets. R package version 2.1.0. https://earth.bsc.es/gitlab/es/startR/
+==== Cite R package ====
+To get the most updated citation text, you can simply use R function "citation". For example,
+    > citation("s2dv")
+    To cite package 's2dv' in publications use:
+      BSC-CNS, An-Chi Ho and Nuria Perez-Zanon (2023). s2dv: A Set of
+      Common Tools for Seasonal to Decadal Verification. R package version
+.0.0. https://CRAN.R-project.org/package=s2dv
+    A BibTeX entry for LaTeX users is
+      @Manual{,
+        title = {s2dv: A Set of Common Tools for Seasonal to Decadal Verification},
+        author = {{BSC-CNS} and An-Chi Ho and Nuria Perez-Zanon},
+        year = {2023},
+        note = {R package version 2.0.0},
+        url = {https://CRAN.R-project.org/package=s2dv},
+      }
+Notice that you may want to change the version number in case the research is done with a previous version.
+==== Cite R ====
+To cite R software along with its base packages, type citation(). The year varies with the R version you use.
+    > citation()
+    To cite R in publications use:
+      R Core Team (2021). R: A language and environment for statistical
+      computing. R Foundation for Statistical Computing, Vienna, Austria.
+      URL https://www.R-project.org/.
+    A BibTeX entry for LaTeX users is
+      @Manual{,
+        title = {R: A Language and Environment for Statistical Computing},
+        author = {{R Core Team}},
+        organization = {R Foundation for Statistical Computing},
+        address = {Vienna, Austria},
+        year = {2021},
+        url = {https://www.R-project.org/},
+      }
+    We have invested a lot of time and effort in creating R, please cite it
+    when using it for data analysis. See also ‘citation("pkgname")’ for
+    citing R packages.
+==== Package publication ====
+Some packages have their own publication, which can also be considered being included.
-  * **ClimProjDiags**
-  N. Pérez-Zanón, and A. Hunter, 2020: ClimProjDiags: Set of tools to compute various climate indices, version 0.1.0. Barcelona Supercomputing Center, R package, https://CRAN.R-project.org/package=ClimProjDiags
   * **CSTools**
   Pérez-Zanón, N., Caron, L.-P., Terzago, S., Van Schaeybroeck, B., Lledó, L., Manubens, N., Roulin, E., Alvarez-Castro, M. C., Batté, L., Bretonnière, P.-A., Corti, S., Delgado-Torres, C., Domínguez, M., Fabiano, F., Giuntoli, I., von Hardenberg, J., Sánchez-García, E., Torralba, V., and Verfaillie, D.: Climate Services Toolbox (CSTools) v4.0: from climate forecasts to climate forecast information, Geosci. Model Dev., 15, 6115–6142, https://doi.org/10.5194/gmd-15-6115-2022, 2022.
-Núria Pérez-Zanón, Louis-Philippe Caron, Carmen Alvarez-Castro, Lauriane Batte, Jost von Hardenberg, Llorenç LLedó, Nicolau Manubens, Eroteida Sánchez-Garcia, Bert van Schaeybroeck, Verónica Torralba and Deborah Verfaillie (2021). CSTools: Assessing Skill of Climate Forecasts on Seasonal-to-Decadal Timescales. R package version 4.0.0. https://CRAN.R-project.org/package=CSTools
   * **CSIndicators**
   Pérez-Zanón, N., Ho, A. Chou, C., Lledó, L., Marcos-Matamoros, R., Rifà, E. and González-Reviriego, N. (2023). CSIndicators: Get tailored climate indicators for applications in your sector. Climate Services. https://doi.org/10.1016/j.cliser.2023.100393
-Pérez-Zanón, N., Chihchung, C. and Lledó, L.: CSIndicators: Sectoral Indicators for Climate Services Based on Sub-Seasonal to Decadal Climate Predictions, CRAN [online] Available from: https://cran.r-project.org/package=CSIndicators, 2021.
   * **s2dverification**
   Nicolau Manubens, Louis-Philippe Caron, Alasdair Hunter, Omar Bellprat, Eleftheria Exarchou, Neven S. Fučkar, Javier Garcia-Serrano, François Massonnet, Martin Ménégoz, Valentina Sicardi, Lauriane Batté, Chloé Prodhomme, Verónica Torralba, Nicola Cortesi, Oriol Mula-Valls, Kim Serradell, Virginie Guemas, Francisco J. Doblas-Reyes, An R package for climate forecast verification, Environmental Modelling & Software, Volume 103, 2018, Pages 29-42, ISSN 1364-8152,https://doi.org/10.1016/j.envsoft.2018.01.018.
-  * **s2dv**
-  BSC-CNS, An-Chi Ho and Núria Pérez-Zanón (2020). s2dv: A Set of Common Tools for Seasonal to Decadal Verification. R package version 0.1.0. https://CRAN.R-project.org/package=s2dv
-  * Given that these packages are using **R environment**, you can also include:
-  R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
-//**Note** that the version of R and packages can be updated, you can check the latest reference by open a terminal and running `citation()` for the R version and `citation("packageName") for each package. We are also working on a manuscript about CSTools.//
+==== Acknowlegement ====
-Finally, it is recommended to add a sentence in **Acknowledgements** section of your manuscripts where readers can check the full list of software used. Here is an example from [[https://journals.ametsoc.org/view/journals/clim/aop/jcliD200138/jcliD200138.xml | 'How Reliable Are Decadal Climate Predictions of Near-Surface Air Temperature?']] (Verfaille et al, 2020):
+You can also choose to put the packages in the acknowledgment section. Here is an example from [[https://journals.ametsoc.org/view/journals/clim/aop/jcliD200138/jcliD200138.xml | 'How Reliable Are Decadal Climate Predictions of Near-Surface Air Temperature?']] (Verfaille et al, 2020):
 //We acknowledge the use of the s2dverification (Manubens et al. 2018), startR (BSC/CNS and Manubens 2020), SpecsVerification (Siegert 2017), CSTools (Pérez-Zanón et al. 2019), ClimProjDiags (BSC/CNS et al. 2020), and boot (Davison and Hinkley 1997; Canty and Ripley 2020) R (R Core Team 2013) software packages.//
+==== Manuscript ====
+It is recommended to add some sentences in the manuscripts where readers can check the full list of software used. For example,
+//All analyses were performed using R Statistical Software (v4.1.2; R Core Team 2021). Temperature data was processed via the R packages: startR (v2.3.0; BSC-CNS etc. 2023), s2dv (v2.0.0; BSC-CNS etc. 2023). The indices were calculated using the R package: CSIndicators (v1.0.1; N. Perez etc. 2023).//
 ===== RStudio-server =====
@@ Line 161: / Line 246: @@
-How to access, what you will find (servers, machines, partitions, modules) and how to open RStudio: check the slides and share them with your colleagues. {{ :tools:what_do_we_know_about_bsc-es_infrastructure_.pdf |slides}}
+How to access, what you will find (servers, machines, partitions, modules) and how to open RStudio: check the slides and share them with your colleagues. {{ :tools:what_do_we_know_about_bsc-es_infrastructure.pdf |slides}}
+How to access BSC Hub and run R from the terminal and using VSCode: {{ :tools:ghr_-_bschub_demo.pdf |}}
@@ Line 195: / Line 282: @@
     module load PROJ/7.2.1-foss-2019b
     module load GEOS/3.7.2-foss-2019b-Python-3.7.4
+(on hub)
+    module load R/4.2.1-foss-2021b
+    module load GDAL/3.5.2-foss-2021b-Python-3.9.6
+    module load PROJ/9.1.0-foss-2021b
+    module load GEOS/3.11.0-GCC-11.2.0
 Note: Avoid to include them in your bashrc, just load them when it is a requirement.

User Tools

Site Tools

Differences

Page Tools