User Tools

Site Tools


tools:eionet-utdretriever

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
tools:eionet-utdretriever [2016/01/27 16:04]
jcuadrad
tools:eionet-utdretriever [2016/12/29 09:09] (current)
kserrade [Repository]
Line 11: Line 11:
 The functionality provided is a retriever of (near real time) air quality observations from stations adhered to EIONET network. The functionality provided is a retriever of (near real time) air quality observations from stations adhered to EIONET network.
  
-The tool is able to connect to EIONET servers, download the required data, parse it, check for validity of observations and store them in the Air Quality Forecast Evaluation (eval_new) database.\\ This tool also inserts new stations in the "STATIONS" table of the DB when a new one is found. The fields automatically inserted are Station code, station name, lat, lon, and height above sea level.+The tool is able to connect to EIONET servers, download the required data, parse it, check for validity of observations and store them in the Air Quality Forecast Evaluation (eval_new) database. **Please note** no CSV or any other output format is provided. All observations are centralized in the database for exploitation through the different systems (like the [[http://www.bsc.es/projects/earthscience/visor/bases_datos/aq/|CALIOPE Visor]]) \\ This tool also inserts new stations in the "STATIONS" table of the DB when a new one is found. The fields automatically inserted are Station code, station name, lat, lon, and height above sea level.
  
 The validity checks performed on the retrieved data are the ones described in this PDF:  The validity checks performed on the retrieved data are the ones described in this PDF: 
Line 43: Line 43:
 The EIONET (up-to-date) air quality data is available for the following geographic areas/countries: The EIONET (up-to-date) air quality data is available for the following geographic areas/countries:
 ^ Code ^ Name ^ Code ^ Name ^ ^ Code ^ Name ^ Code ^ Name ^
-ATAustria | IE| Ireland | +ADAndorra | IE| Ireland | 
-BEBelgium | LT| Lithuania | +ATAustria | LT| Lithuania | 
-DEGermany | LU| Luxembourg |+BEBelgium | LU| Luxembourg 
 +| DE| Germany | LV| Latvia |
 | DK| Denmark | MK| Macedonia | | DK| Denmark | MK| Macedonia |
 | ES| Spain | MT| Malta | | ES| Spain | MT| Malta |
Line 54: Line 55:
 | HR| Croatia | SE| Sweden | | HR| Croatia | SE| Sweden |
 | HU| Hungary | SI| Slovenia | | HU| Hungary | SI| Slovenia |
 +| |  | SK| Slovakia |
  
-To see the **current status on data delivery**, which countries are delivering data and what they deliver, please see this [[https://tableau.discomap.eea.europa.eu/t/Aironline/views/Airquality_E2a_monitoring/DashboardE2a?:embed=y&:showShareOptions=true|E2a/UTD Air quality - primary pollutants delivery LIVE report]]. The detailed report (in use until Dec/2015) can be found here: [[http://discomap.eea.europa.eu/report/eMonitoring/CurrentStatusE2a|live report]]. \\+To see the **current status on data delivery**, which countries are delivering data and what they deliver, please see this **[[https://tableau.discomap.eea.europa.eu/t/Aironline/views/Airquality_E2a_monitoring/DashboardE2a?:embed=y&:showShareOptions=true|E2a/UTD Air quality - primary pollutants delivery LIVE report]]**. The former, more detailed report can be found here: [[http://discomap.eea.europa.eu/report/eMonitoring/CurrentStatusE2a|live report]] (in use until Dec/2015). \\
  
 For further information about stations definitions used by EIONET please refer to: [[tools:EIONET-UTDretriever:Stations|EIONET definitions for AQ Stations]]  For further information about stations definitions used by EIONET please refer to: [[tools:EIONET-UTDretriever:Stations|EIONET definitions for AQ Stations]] 
 +
 +Please note that EIONET reports the different network_timezone for each station in which the reported observations are. This tool automatically translates all observations to UTC. All observations stored in the eval_new database are in UTC. \\
 +** Since 20/apr/2016 EIONET is reporting the metadata for the time zone of observations in Spain, Lithuania, Macedonia and Slovenia (that was previously missing). Therefore, all observations since 20/apr/2016 should be as correct as the information provided by the Member States to EIONET. To date, 05/may/2016 we are still troubleshooting (in contact with Generalitat de Catalunya and EIONET) a discrepancy of 1h of difference in observations for Catalunya. ** \\
 +Until 20/apr/2016 it was assumed that observations were in UTC, when no metadata was provided.
  
 Please refer to the "Usage" section for **configuration options** and to see details about pollutant acronyms/notations and time spans of observations. Please refer to the "Usage" section for **configuration options** and to see details about pollutant acronyms/notations and time spans of observations.
Line 129: Line 135:
  
 == Further documentation == == Further documentation ==
-**Internal request fields (to the EIONET service) used in this tool can be found here: http://discomap.eea.europa.eu/map/fme/doc/UTDAirQualityDownloadGuide.pdf** \\ +  * Slides presenting the tool, including information about the number of stations, the rationale behind the different config files and for the chosen update patterns used in the cron, etc. {{tools:eionet-utdretriever:20160209_EIONET-UTDRetriever-Updated.pdf|EIONET-UTDRetriever-Updated_ToolSummary.pdf}} 
-More technical documentation (Sphinx-generated) can be found under the directory docs in the repository.+  * **Internal request fields (to the EIONET service) used in this tool** can be found here: http://discomap.eea.europa.eu/map/fme/doc/UTDAirQualityDownloadGuide.pdf \\ 
 +  More technical documentation (Sphinx-generated) can be found under the directory docs in the repository. 
  
  
 ==== Repository ==== ==== Repository ====
  
-The link to the GIT repository is:\\ <code>https://earth.bsc.es/gitlab/jcuadrad/EIONET.git</code>+The link to the GIT repository is:\\ <code>https://earth.bsc.es/gitlab/es/EIONET.git</code>
  
  
tools/eionet-utdretriever.1453910696.txt.gz · Last modified: 2016/01/27 16:04 by jcuadrad