## Deployment of startR

This section contains the information on system requirements and the steps to set up such requirements. Note that `startR` can be used for two different purposes, either only retrieving data locally, or retrieving plus processing data on a distributed HPC. The requirements for each purpose are detailed in separate sections.

### Deployment steps for retrieving data locally

1. Install netCDF-4 if retrieving data from NetCDF files (only file format supported by now):
    - zlib (>= 1.2.3) and HDF5 (>= 1.8.0-beta1) are required by netCDF-4
    - Steps for installation of netCDF-4 are detailed in https://www.unidata.ucar.edu/software/netcdf/docs/getting_and_building_netcdf.html

2. Install R (>= 2.14.1)

3. Install the required R packages
    - Installing `startR` will trigger the installation of the other required packages
```r
devtools::install_git('https://earth.bsc.es/gitlab/es/startR')
```
    - Among others, the bigmemory package will be installed.
    - If loading and processing NetCDF files (only file format supported by now), install the easyNCDF package.
    - If planning to interpolate the data with CDO (either by using the `transform` parameter in `startR::Start`, or by using `s2dverification::CDORemap` in the workflow specified to `startR::Compute`), install s2dverification (>= 2.8.4) and CDO (version 1.6.3 tested). CDO is not available for Windows.

A local or remote file system or THREDDS/OPeNDAP server providing the data to be retrieved must be accessible.

### Deployment steps for processing data on distributed HPCs

For processing the data on a distributed HPC (cluster of multi-processor, multi-core nodes), your workstation, an optional EC-Flow host node and a HPC login node with acces to the HPC nodes, should all be accessible in your network.

All machines must be UNIX-based, with the "hostname", "date", "touch" and "sed" commands available.

1. Set up passwordless, userless ssh access
    - at least from your workstation to the HPC login node
    - if possible, also from the HPC login node to your workstation

2. Install the following libraries on your workstation:
    - ssh
    - scp
    - rsync (>= 3.0.6)
    - EC-Flow (>= 4.9.0)

3. If you are using a separate EC-Flow host node to control your EC-Flow workflows (optional), install EC-Flow (>= 4.9.0) on the EC-Flow host node

4. Install the following libraries on the HPC login node:
    - ssh
    - scp
    - rsync (>= 3.0.6)
    - EC-Flow (>= 4.9.0), as a Linux Environment Module (optional)
    - Job scheduler (Slurm, PBS or LSF) to distribute the workload across HPC nodes

5. Make sure the following requirements are fulfilled by all HPC nodes:
    - netCDF-4 is installed, if loading and processing NetCDF files (only supported format by now)
    - R (>= 2.14.1) is installed as a Linux Environment Module
      - the startR package is installed
      - if using CDO interpolation, the s2dverification package and CDO 1.6.3 are installed
      - any other R packages required by the `startR::Compute` workflow are installed
    - any other Environment Modules used by the `startR::Compute` workflow are installed
    - a shared file system (with a unified access point) or THREDDS/OPeNDAP server is accessible across HPC nodes and HPC login node, where the necessary data can be uploaded from your workstation. A file system shared between your workstation and the HPC is also supported and advantageous. Use of a data transfer service between the workstation and the HPC is also supported under specific configurations.
