\name{LoadConfigurationFile}
\alias{LoadConfigurationFile}
\title{
  Load Dataset Location Data Into An Environment
}
\description{
This function loads all the data contained in the configuration file specified as parameter 'configfile' in the function Load() of s2dverification.
Extends the environment given as parameter with the variables and functions needed for the configuration file mechanism to work.

How the configuration file works: 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It contains two lists and three tables.
Each of these have a header that starts with '!!'. These are key lines and should not be removed or reordered.
Lines starting with '#' and blank lines will be ignored.

The first list should contain all the supported 2-dimensional variables.
The second list should contain all the supported global mean variables.
Regular expressions[1] can be used in these lists.

The first table contains information about experiments.
The second table contains information about file-per-ensemble experiments (those experiments of which the data for each member is stored in a separate file).
The third table contains information about observations.

Each table entry is a list of comma-separated elements. 
The two first are part of a key that is associated to a value formed by the other elements.
The key elements are a dataset name and a variable name.
The value elements are the dataset main path, dataset file path, grid (unless it is an observation table entry) and the variable name inside the .nc file (value).

Given a dataset name and a variable name, a full path is obtained concatenating the main path and the file path with a slash (/) in between.
Also the grid (in the case it is an experiment) and the nc variable name are obtained.

Any of the elements in the keys can contain regular expressions that will cause matching for sets of dataset names or variable names.

The dataset path and file path can contain shell globbing expressions[2] that will cause matching for sets of paths when fetching the file in the full path.

Any of the elements in the value can contain variables that will be replaced to an associated string.
Variables can be defined only immediately above any of the tables header. 
The pattern of a variable definition is
  VARIABLE_NAME = VARIABLE_VALUE
and can be accessed from within the table values or from within the variable values as
  $VARIABLE_NAME$
For example:
  FILE_NAME = tos.nc
  !!table_of_experiments
  ecmwf, tos, /path/to/dataset, $FILE_NAME$
There are some reserved variables that will offer information about the store frequency, the current startdate Load() is fetching, etc:
  $START_DATE$, $STORE_FREQ$
  for file-per-ensemble datasets: $ENSEMBLE_NUMBER$
  for observations: $YEAR$, $MONTH$, $DAY$
Additionally, from an element in an entry value you can access the other elements of the entry as:
  $EXP_NAME$, $VAR_NAME$, $EXP_MAIN_PATH$, $EXP_FILE_PATH$, $GRID$, $VAR_NAME$
and in an observation as:
  $OBS_NAME$, $VAR_NAME$, $OBS_MAIN_PATH$, $OBS_FILE_PATH$, $VAR_NAME$

The entries in a table will be grouped in 4 levels of specificity:
1- General entries:
   · the key dataset name and variable name are both an asterisk (*) that will cause matching for any pair of dataset and variable names
   Example:  *, *, /dataset/main/path, file/path, grid, nc_var_name
2- Dataset entries:
   · the key variable name is an asterisk
   Example:  ecmwf, *, /dataset/main/path, file/path, grid, nc_var_name
3- Variable entries:
   · the key dataset name is an asterisk
   Example:  *, tos, /dataset/main/path, file/path, grid, nc_var_name
4- Specific entries:
   · both key values are specified
   Example:  ecmwf, tos, /dataset/main/path, file/path, grid, nc_var_name

Given a pair of dataset name and variable name for which we want to know the full path, all the rules that match will be applied from more general to more specific.
If there is more than one entry per group that match a given key pair, these will be applied in the order of appearance in the configuration file (top to bottom).

An asterisk (*) in any value element will be interpreted as 'leave it as is or take the default value if yet not defined'.
The default values are defined in the following reserved variables:
  $DEFAULT_EXP_MAIN_PATH$, $DEFAULT_EXP_FILE_PATH$, $DEFAULT_GRID$, $DEFAULT_NC_VAR_NAME$, $DEFAULT_OBS_MAIN_PATH$, $DEFAULT_OBS_FILE_PATH$
Trailing asterisks in an entry are not mandatory. For example
  ecmwf, *, /dataset/main/path, *, *, *
will have the same effect as
  ecmwf, *, /dataset/main/path

A double quote (") in any key or value element will be interpreted as 'fill in with the same value as the entry above'.

The pattern to obtain the full path by concatenating the dataset main path and the dataset file path can be adjusted in the variables EXP_FULL_PATH and OBS_FULL_PATH, which are defined by default as
  EXP_FULL_PATH = $EXP_MAIN_PATH$/$EXP_FILE_PATH$
  OBS_FULL_PATH = $OBS_MAIN_PATH$/$OBS_FILE_PATH$

[1] http://tldp.org/LDP/abs/html/globbingref.html
[2] https://stat.ethz.ch/R-manual/R-devel/library/base/html/regex.html
}
\usage{
  LoadConfigurationFile(environment)
}
\arguments{
  \item{environment}{
    R environment to extend with the configuration file mechanism variables and functions.
  }
}
\value{
  Extended environment with the configuration file mechanism variables and functions.
}
\examples{
  configfile <- system.file("config", "template.conf", package = "s2dverification")
  LoadConfigurationFile(environment())
}
\author{
  History:
  0.1  -  2014-12  (N. Manubens, \email{nicolau.manubens@ic3.cat})  -  First version
}
\keyword{datagen}
