tools:actools:cmaq_reduced []

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== CMAQ Reduced ======
Under the conventions of the {{:library:external:technical_memoranda:bsc-ac-2016-002-data_storage_protocol_v1.pdf|BSC-AC Technical Memorandum 2016-002}} this tools makes the reduced version of the CMAQ (CCTM) outputs.

This a tool developed specifically for the CMAQ outputs, it is developed in Python 2.7.x language and use many of the Python libraries to make it easy.

===== Content =====
This tool is composed of 3 elements
  * Code
  * Configuration file
  * Reduced specifications

==== Code ====
//cmaq_reduce.py// is fully in Python 2.7.x language and uses some libraries:
  * ConfigParser -> To read the configuration file.
  * pandas -> To read the CSV as a DataFrame.
  * numpy -> To do the calculations of the information.
  * datetime -> To use correct dates.
  * netCDF4 -> To use NetCDF files.

All the functions of the script are documented following the conventions. Feel free to inspect them.

To run the script is needed two more files to specify What to do (Configuration file) and How to do it (Reduced specifications).

==== Configuration file ====
//cmaq_reduce.conf// is the configuration file needed to run the script. You must give the configuration file as an argument <code>./cmaq_reduced.py cmaq_reduced.conf</code>

This configuration file is divided in 3 sections and all of the parameters are commented to clarify them.

All the sections will be explained later, but first you have to take in account that normally the paths where are stored the MCIP and CCTM outputs are not statics.

The way to do these dynamic paths is by sustitutions:
  * <date>   -> YYYYMMDDHH
  * <dateJ>  -> YYYYJJJ
  * <domain> -> domain (IP, EU, ...)
example:
   start_date = 30/12/2016
   domain = IP4
   cmaq_out_dir = /gpfs/projects/bsc32/bsc32359/CMAQ-mat/FORECAST/AQ/F-<domain>/OUT
   mcip_folder = MCIP/OUTPUT_MCIP_<dateJ>
   cctm_folder = CCTM/<dateJ>

result:
   mcip_directory -> /gpfs/projects/bsc32/bsc32359/CMAQ-mat/FORECAST/AQ/F-IP4/OUT/MCIP/OUTPUT_MCIP_2016366
   cctm_directory -> /gpfs/projects/bsc32/bsc32359/CMAQ-mat/FORECAST/AQ/F-IP4/OUT/CCTM/2016366


== General Section ==
  * start_date -> Starting date to generate reduceds. All the dates must to be in the format "DD/MM/YYYY".  If you want to run the current day start_date = TODAY
  * end_date -> End date to generate the reduceds. If you want to do only one day it have to be the same date as the start_date.
  * exp_id -> ID of the experiment. For Caliope it is CALIOPE/<domain> 
  * domain -> Domain to do the reduced. This value will be used to substitute <domain> on the paths
  * csv_path -> Complete path to the CSV that contains the info of the reduceds to do. This CSV will be explained bellow.

== Inputs Section ==
  * cmaq_out_dir -> Common path of all the outputs of CMAQ.
  * mcip_folder -> Relative path from the cmaq_out_dir to the MCIP directory.
  * cctm_folder -> Relative path from the cmaq_out_dir to the CCTM directory.

== Outputs Section ==
  * output_directory -> Path to store the outputs. In this path will be generated the conventions folders to store all the outputs.


==== Reduced specifications ====
The General section of the configuration file have the csv_path parameter that have to point to this file.

As you are supposing these specifications are under a CSV file like this:
<code>
var,var_cmaq,filename,factor,unit,timeint_stat
sconco3,O3,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h 8h_max day_mean day_max
sconcco,CO,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h 8h_max day_mean day_max
sconcno2,NO2,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h 8h_max day_mean day_max
sconcso2,SO2,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h 8h_max day_mean day_max
sconcno,NO,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h
sconchono,HONO,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h
sconcnh3,NH3,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h
sconchno3,HNO3,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,1,ppm,1h
pm2p5,ASO4J ASO4I ANH4J ANH4I ANO3J ANO3I AORGPAJ AORGPAI AALKJ AXYL1J AXYL2J AXYL3J ATOL1J ATOL2J ATOL3J ABNZ1J ABNZ2J ABNZ3J AOLGAJ ATRP1J ATRP2J AISO1J AISO2J AISO3J ASQTJ AOLGBJ AORGCJ AECJ AECI A25J ANAJ ACLJ ACLI,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h day_mean day_max
pm2p5so4,ASO4I ASO4J,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5ppm,A25J AECJ AECI AORGPAJ AORGPAI AECI AECJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5oc,AORGPAJ AORGPAI AALKJ AXYL1J AXYL2J AXYL3J ATOL1J ATOL2J ATOL3J ABNZ1J ABNZ2J ABNZ3J AOLGAJ ATRP1J ATRP2J AISO1J AISO2J AISO3J ASQTJ AOLGBJ AORGCJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5no3,ANO3I ANO3J,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5nh4,ANH4I ANH4J,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5na,ANAJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5cl,ACLJ ACLI,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5bc,AECI AECJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm2p5du,,,,,
pm2p5soa,AALKJ AXYL1J AXYL2J AXYL3J ATOL1J ATOL2J ATOL3J ABNZ1J ABNZ2J ABNZ3J AOLGAJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10,ASO4J ASO4I ANH4J ANH4I ANO3J ANO3I AORGPAJ AORGPAI AALKJ AXYL1J AXYL2J AXYL3J ATOL1J ATOL2J ATOL3J ABNZ1J ABNZ2J ABNZ3J AOLGAJ ATRP1J ATRP2J AISO1J AISO2J AISO3J ASQTJ AOLGBJ AORGCJ AECJ AECI A25J ANAJ ACLJ ACLI ASO4K ANH4K ANO3K ANAK ACLK ACORS ASOIL,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h day_mean day_max
pm10so4,ASO4I ASO4J,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10ppm,A25J AECJ AECI AORGPAJ AORGPAI ACORS ASOIL AECI AECJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10oc,AALKJ AXYL1J AXYL2J AXYL3J ATOL1J ATOL2J ATOL3J ABNZ1J ABNZ2J ABNZ3J AOLGAJ ATRP1J ATRP2J AISO1J AISO2J AISO3J ASQTJ AOLGBJ AORGCJ AORGPAJ AORGPAI,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10no3,ANO3I ANO3J ANO3K,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10nh4,ANH4I ANH4J ANH4K,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10na,ANAJ ANAK,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10cl,ACLJ ACLI ACLK,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10bc,AECI AECJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
pm10du,,,,,
pm10soa,AALKJ AXYL1J AXYL2J AXYL3J ATOL1J ATOL2J ATOL3J ABNZ1J ABNZ2J ABNZ3J AOLGAJ ATRP1J ATRP2J AISO1J AISO2J AISO3J ASQTJ AOLGBJ AORGCJ,CCTM_FORECAST_<domain>.ACONC.FORECAST_<domain>,0.00000001,kgm-3,1h
vmro3,O3,CCTM_FORECAST_<domain>.CONC.FORECAST_<domain>,1,mol/mol,1h
vmrno2,NO2,CCTM_FORECAST_<domain>.CONC.FORECAST_<domain>,1,mol/mol,1h
vmrso2,SO2,CCTM_FORECAST_<domain>.CONC.FORECAST_<domain>,1,mol/mol,1h
t2,TEMP2,METCRO2D_<domain>,1,K,1h
pblh,PBL,METCRO2D_<domain>,1,m,1h
wspd10,WSPD10,METCRO2D_<domain>,1,ms-1,1h
sdir10,WDIR10,METCRO2D_<domain>,1,deg,1h
psfc,PRSFC,METCRO2D_<domain>,1,Pa,1h
ustar,USTAR,METCRO2D_<domain>,1,ms-1,1h
t,TA,METCRO3D_<domain>,1,K,1h
pint,PRES,METCRO3D_<domain>,1,Pa,1h
z,ZH,METCRO3D_<domain>,1,m,1h
zf,ZF,METCRO3D_<domain>,1,m,1h
</code>

In the first line it has the header that specifies the content of the following lines.
<code>
var,var_cmaq,filename,factor,unit,timeint_stat
</code>
  * var -> Name of the output variable following the conventions.
  * var_cmaq -> Variable or List of variables of the CMAQ file to reduce.
  * filename -> CMAQ file to reduce.
  * factor -> Conversion factor to store the value in the correct units.
  * unit -> Units to save the outputs.
  * timeint_stat -> Type (or list of types) of the reduceds to do. the Reduced types will be explained in the REDUCED TYPES SECTION.

===== Reduced types =====
All the output types are saved following the NetCDF conventions with the lat lon localization and the datetime.
==== 1h ====
This output type is directly the hourly values.

The output type will have the same time steps of the input.
==== 8h_max ====
That's the most complex reduced type.

The result is one value for each cell and each day.
This result is the maximum daily 8 hour mean.

DIRECTIVE 2008/50/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL

Maximum daily eight-hour mean: the maximum daily eight-hour mean concentration shall be selected by examining eight-hour running averages, calculated from hourly data and updated each hour. Each eight -hour average so calculated shall be assigned to the day on which it ends. i.e. the first calculation period for any one day will be the period from 17:00 on the previous day to 01:00 on that day; the last calculation  period for any one day will be the period from 16:00 to 24:00 on the day.

==== day_mean ====
The mean of each day
==== day_max ====
The maximum value of each day
User Tools

Site Tools

Page Tools
