User Tools

Site Tools

Trace:
working_groups:ukurbangroup

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
working_groups:ukurbangroup [2022/09/19 11:28]
acriado 		working_groups:ukurbangroup [2022/09/23 09:10] (current)
acriado
Line 27: Line 27:
 module load RStudio/1.1.463-foss-2015a module load RStudio/1.1.463-foss-2015a
 module load R/3.6.1-foss-2015a-bare module load R/3.6.1-foss-2015a-bare
-module load GDAL/2.2.1-foss-2015a-GEOS-3.8.0 
-module load PROJ/6.1.1-foss-2015a 
-module load GEOS/3.8.0-foss-2015a 
-module load HDF5/1.10.5-foss-2015a 
 rstudio & rstudio &
 </code> </code>
Line 40: Line 36:
 To launch it in the workstation, you have to configure your //bashrc//: To launch it in the workstation, you have to configure your //bashrc//:
  
-  * <code bash>+  * Open your bashrc: 
 +<code bash>
 vi ~/.bashrc vi ~/.bashrc
- </code> +</code> 
-Open the Rstudio application:+ 
 +  * Save the modules:
 <code bash> <code bash>
-  * source ~/.bashrc+module load RStudio/1.1.463-foss-2015a 
 +module load R/3.6.1-foss-2015a-bare
 </code> </code>
-Open the Rstudio application:+ 
 +  * Save your bashrc:
 <code bash> <code bash>
-  * Launch the script :  +source ~/.bashrc
 </code> </code>
-Rscript [script]+  * Launch the script:
 <code bash> <code bash>
 +Rscript [script]
 +</code>
  
 ===== First Steps : understanding the scripts =====  ===== First Steps : understanding the scripts ===== 
Line 78: Line 80:
 Now we are going to see what is and the implication of each of the items that have to be filled in the configuration file: Now we are going to see what is and the implication of each of the items that have to be filled in the configuration file:
  
-  * **//UniversalKriging_path//** : (a folder path) \\ this will be the main folder, the one that is created after copying the repository. It must contain the sub-folder with all the archives required, which would be **//general//**: +  * **//repository_path//** : (a folder path) \\ this would be the folder where the user has copied the repository. It will contain the sub-folder with all the archives required, which will be **//general//**.
-    * **//general//** : (a folder path) \\ this path must be the one where all the archives copied from the repository are kept, and it is included in the **//UniversalKriging_path//**. +
  
 +  * **//UniversalKriging_path//** : (a folder path) \\ this will be the folder with all the results and data created by the code. 
 +   
   * **//year//** : (a number) \\ this is referred to the chosen year. The available years are 2017, 2019, 2020 and 2021.   * **//year//** : (a number) \\ this is referred to the chosen year. The available years are 2017, 2019, 2020 and 2021.
     *  **//exp_caliope_urban_path//** : (a character). Each character refers to an experiment, depending on the chosen year (2017: old/a3xa ; 2019: a4er ; 2020: a4eu ; 2021: a4hm, a4hl). \\ This parameter is needed to use the CALIOPE-Urban output linked to the year of application. In the first application, the output is copied into the user's directory from one of the experiments mentioned. __The user only should fill the character in terms of the chosen year, but the experiments should be the mentioned ones if the correct version of CALIOPE-Urban will be used__. This has to be written between"".     *  **//exp_caliope_urban_path//** : (a character). Each character refers to an experiment, depending on the chosen year (2017: old/a3xa ; 2019: a4er ; 2020: a4eu ; 2021: a4hm, a4hl). \\ This parameter is needed to use the CALIOPE-Urban output linked to the year of application. In the first application, the output is copied into the user's directory from one of the experiments mentioned. __The user only should fill the character in terms of the chosen year, but the experiments should be the mentioned ones if the correct version of CALIOPE-Urban will be used__. This has to be written between"".
-    * **//GHOST_no2_path//** : (a folder path: "/gpfs/projects/bsc32/AC_cache/obs/ghost/EEA_AQ_eReporting/1.4/hourly/sconcno2/") \\ this is the path needed to use the observations coming from the monitoring stations used. The observations will be copied into the user's directory from that path in the first application. __The user should not change this path if the GHOST observations are used.__+   
 +  * **//obs_monitoring_data//** : (//TRUE// or //FALSE//). If //TRUE//, the user will use the diagnosis mode with the 12 monitoring stations considered, and **//other_obs_file//** must be //FALSE//. If //FALSE//, the user must provide a new observations file in the path **//other_obs_file//**, and it must be a list (each entry is an observational point) and the name of the values has to be //obs_value//. This option is under revision, so we recommend not touching these parameters and setting **//obs_monitoring_data//** = //TRUE// and **//other_obs_file//** = //FALSE//.  
 +  * **//GHOST_no2_path//** : (a folder path: "/gpfs/projects/bsc32/AC_cache/obs/ghost/EEA_AQ_eReporting/1.4/hourly/sconcno2/") \\ this is the path needed to use the observations coming from the monitoring stations used. The observations will be copied into the user's directory from that path in the first application. __The user should not change this path if the GHOST observations are used.__
  
   * **//full_year//** : (//TRUE// or //FALSE//) \\ if the user wants to bias-correct the whole year, this parameter would be //TRUE//, then //date_begin// and //date_end// will be FALSE. If **//full_year//** is //FALSE//, the period chosen cannot involve two different years. In other cases:    * **//full_year//** : (//TRUE// or //FALSE//) \\ if the user wants to bias-correct the whole year, this parameter would be //TRUE//, then //date_begin// and //date_end// will be FALSE. If **//full_year//** is //FALSE//, the period chosen cannot involve two different years. In other cases: 
Line 92: Line 96:
    * **//UK_mode//** : (//UK1// or //UK2//) \\ this parameter is referred to the usage or not of the microscale-LUR model as the second covariate. If //UK_mode// is //UK1//, only CALIOPE-Urban is used as the covariate. If //UK_mode// is //UK2//, both CALIOPE-Urban and the microscale-LUR model will be the covariates. This has to be written between [ ]. Only if the //UK_mode// is //UK2// the microscale-LUR model will be used.    * **//UK_mode//** : (//UK1// or //UK2//) \\ this parameter is referred to the usage or not of the microscale-LUR model as the second covariate. If //UK_mode// is //UK1//, only CALIOPE-Urban is used as the covariate. If //UK_mode// is //UK2//, both CALIOPE-Urban and the microscale-LUR model will be the covariates. This has to be written between [ ]. Only if the //UK_mode// is //UK2// the microscale-LUR model will be used.
    * **//application//** : (all possible combination using the following items; //UK//, //cross// , //UK_max//, //cross_max//, //UK_mean//, //uncertainty// ) \\ this is referred to the application of the methodology that we want. This/They has/have to be written between [ ]:     * **//application//** : (all possible combination using the following items; //UK//, //cross// , //UK_max//, //cross_max//, //UK_mean//, //uncertainty// ) \\ this is referred to the application of the methodology that we want. This/They has/have to be written between [ ]: 
-   * //UK//: Universal Kriging hourly correction (the default). +     - //UK//: Universal Kriging hourly correction (the default). 
-   * //cross//: If we want to apply the Leave-One-Out Cross-Validation, and get the results only at the monitoring stations. +     - //cross//: If we want to apply the Leave-One-Out Cross-Validation, and get the results only at the monitoring stations. 
-   * //UK_max// and //cross_max//: They are the same, as mentioned before, but instead of the hourly application, the maximum daily one.  +     - //UK_max// and //cross_max//: They are the same, as mentioned before, but instead of the hourly application, the maximum daily one.  
-   * //UK_mean//: This is for computing the daily and annual mean corrected concentrations. Nothing will occur if the //UK// application is not performed before applying.  +     - //UK_mean//: This is for computing the daily and annual mean corrected concentrations. Nothing will occur if the //UK// application is not performed before applying.  
-   * //uncertainty// is for calculating the daily mean variances, the annual mean variance and the relative uncertainty associated with the application. +     - //uncertainty// is for calculating the daily mean variances, the annual mean variance and the relative uncertainty associated with the application. 
  
    
Line 104: Line 108:
  
 As a resume, notice that the user only has to do the following about the configuration file: As a resume, notice that the user only has to do the following about the configuration file:
-  * Choosing you own **//UniversalKriging_path//** and referring to it the path **//general//**.+  * Choosing you own **//repository_path//** and **//UniversalKriging_path//**.
   * Choosing one of the **//year//**s available, with the appropriate **//exp_caliope_urban_path//**, and choosing between a whole year correction or not.   * Choosing one of the **//year//**s available, with the appropriate **//exp_caliope_urban_path//**, and choosing between a whole year correction or not.
   * Choosing the Universal Kriging mode in terms of the covariates, the application, whether plotting and doing the evaluation, and the cores to submit the job.   * Choosing the Universal Kriging mode in terms of the covariates, the application, whether plotting and doing the evaluation, and the cores to submit the job.
Line 149: Line 153:
   * //#SBATCH --time=48:00:00// : if the user wants to apply for the first time (the secondary operations are not made yet) a whole year correction using all the applications available, with approximately between 30 and 50 cores for the parallelization. In this case, due to the high computational requirements of the job, we also recommend constraining the memory to "high": //#SBATCH --constraint=highmem//. For example, calculating the daily mean values of caliope urban during a whole year takes 6h of computational time using 50 cores.   * //#SBATCH --time=48:00:00// : if the user wants to apply for the first time (the secondary operations are not made yet) a whole year correction using all the applications available, with approximately between 30 and 50 cores for the parallelization. In this case, due to the high computational requirements of the job, we also recommend constraining the memory to "high": //#SBATCH --constraint=highmem//. For example, calculating the daily mean values of caliope urban during a whole year takes 6h of computational time using 50 cores.
   * //#SBATCH --time=01:00:00// : if the user wants to apply, without secondary operations and during a whole year, the applications of //UK// and //cross//, with approximately 30 cores for the parallelization.   * //#SBATCH --time=01:00:00// : if the user wants to apply, without secondary operations and during a whole year, the applications of //UK// and //cross//, with approximately 30 cores for the parallelization.
 +
 +The user has to charge the model in the //bashrc// (referred to the machine):
 +  * Open your bashrc:
 +<code bash>
 +vi ~/.bashrc
 +</code>
 +
 +  * Save the modules:
 +<code bash>
 +module load R
 +</code>
 +
 +  * Save your bashrc:
 +<code bash>
 +source ~/.bashrc
 +</code>
 +
  
 To submit the job regarding this methodology, the user only has to launch its own //[name].sh// archive with the Rscript //kriging_repository.R// followed by the configuration file. As an example: To submit the job regarding this methodology, the user only has to launch its own //[name].sh// archive with the Rscript //kriging_repository.R// followed by the configuration file. As an example:
working_groups/ukurbangroup.1663586928.txt.gz · Last modified: 2022/09/19 11:28 by acriado

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki