Commit 111a22ff authored by erifarov's avatar erifarov
Browse files

Changed queue_host to nord4 login

parent 6a125f06
Pipeline #7316 passed with stage
in 69 minutes and 11 seconds
# Practical guide for processing large data sets with startR
This guide includes explanations and practical examples for you to learn how to use startR to efficiently process large data sets in parallel on the BSC's HPCs (Nord3v2, CTE-Power 9, Marenostrum 4, ...). See the main page of the [**startR**](README.md) project for a general overview of the features of startR, without actual guidance on how to use it.
This guide includes explanations and practical examples for you to learn how to use startR to efficiently process large data sets in parallel on the BSC's HPCs (Nord3-v2, CTE-Power 9, Marenostrum 4, ...). See the main page of the [**startR**](README.md) project for a general overview of the features of startR, without actual guidance on how to use it.
If you would like to start using startR rightaway on the BSC infrastructure, you can directly go through the "Configuring startR" section, copy/paste the basic startR script example shown at the end of the "Introduction" section onto the text editor of your preference, adjust the paths and user names specified in the `Compute()` call, and run the code in an R session after loading the R and ecFlow modules.
......@@ -53,7 +53,7 @@ Afterwards, you will need to understand and use five functions, all of them incl
- **Compute()**, for specifying the HPC to be employed, the execution parameters (e.g. number of chunks and cores), and to trigger the computation
- **Collect()** and the **EC-Flow graphical user interface**, for monitoring of the progress and collection of results
Next, you can see an example startR script performing the ensemble mean of a small data set on an HPC cluster such as Nord3v2 or CTE-Power9, for you to get a broad picture of how the startR functions interact and the information that is represented in a startR script. Note that the `queue_host`, `temp_dir` and `ecflow_suite_dir` parameters in the `Compute()` call are user-specific.
Next, you can see an example startR script performing the ensemble mean of a small data set on an HPC cluster such as Nord3-v2 or CTE-Power9, for you to get a broad picture of how the startR functions interact and the information that is represented in a startR script. Note that the `queue_host`, `temp_dir` and `ecflow_suite_dir` parameters in the `Compute()` call are user-specific.
```r
library(startR)
......@@ -679,7 +679,7 @@ As mentioned above in the definition of the `cluster` parameters, it is strongly
The `Compute()` call with the parameters to run the example in this section on the BSC ES fat nodes is provided below (you will need to adjust some of the parameters before using it). As you can see, the only thing that needs to be changed to execute startR on a different HPC is the definition of the `cluster` parameters.
The `cluster` configuration for the fat nodes, CTE-Power 9, Marenostrum 4, Nord3, Minotauro and ECMWF cca/ccb are all provided at the very end of this guide.
The `cluster` configuration for the fat nodes, CTE-Power 9, Marenostrum 4, Nord3-v2, Minotauro and ECMWF cca/ccb are all provided at the very end of this guide.
```r
res <- Compute(wf,
......@@ -1072,10 +1072,10 @@ cluster = list(queue_host = 'mn2.bsc.es',
)
```
### Nord3
### Nord3-v2
```r
cluster = list(queue_host = 'nord1.bsc.es',
cluster = list(queue_host = 'nord4.bsc.es',
queue_type = 'lsf',
data_dir = '/gpfs/projects/bsc32/share/startR_data_repos/',
temp_dir = '/gpfs/scratch/bsc32/bsc32473/startR_hpc/',
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment