diff --git a/README.md b/README.md index 7ddc808fc2db41958dd0fa9fa24f479455fda5cf..2d717e556197c7400df6202cc6b38dffcbfa74ff 100644 --- a/README.md +++ b/README.md @@ -2,19 +2,23 @@ This repository contains the scripts related to the Observation Apps - components for ground-based (SYNOP), radiosounding (TEMP), and satellite (AMSU-A) obs data) of the Climate Adaptation Digital Twin (Climate DT). The development is done in the frameworks of the Destination Earth initiative. # Description -The OBS application is currently being developed as a set of scripts (written in bash, fortran, python): -1. for pre-processing (with operators of cdo, Climate Data Operators, software) gsv_extracted modeled data (over latitude-longitude domain) for selected meteorological variables into hourly time-slices (00...23 UTCs); -2. for extracting (using sql with odp_api software) selected meteorological variables at hourly time-slices for observations from: (i) ground-based synoptical stations at fixed at the surface geographical locations/points (i.e., 2D: latitude, longitude); (ii) vertical radiosounding of the atmosphere at changing locations/points (3D: latitude, longitude, single pressure level), and (iii) satellite at changing locations/ points (3D: latitude, longitude, multiple pressure levels); -3. for extracting and interpolating (using operators of cdo software - for synop; & expecting/using polytope - for radiosounding and satellite) modeled data for the same time-slices for same selected meteorological variables into corresponding locations/points of (synop, radiosounding, satellite) observations, and adding (using import with odb_api) such data to ODB; +The OBS application is currently being developed as a set of scripts (written in `bash`, `fortran`, `python`): +1. for pre-processing (with operators of `cdo`, Climate Data Operators, software) gsv_extracted modeled data (over latitude-longitude domain) for selected meteorological variables into hourly time-slices (00...23 UTCs); +2. for extracting (using `sql` with `odp_api` software) selected meteorological variables at hourly time-slices for observations from: (i) ground-based synoptical stations at fixed at the surface geographical locations/points (i.e., 2D: latitude, longitude); (ii) vertical radiosounding of the atmosphere at changing locations/points (3D: latitude, longitude, single pressure level), and (iii) satellite at changing locations/ points (3D: latitude, longitude, multiple pressure levels); +3. for extracting and interpolating (using `cdo` operators - for synop; & expecting/using `polytope` - for radiosounding and satellite) modeled data for the same time-slices for same selected meteorological variables into corresponding locations/points of (synop, radiosounding, satellite) observations, and adding (using `import` with `odb_api`) such data to ODB; 3. for calculating/ producing relevant statistics such as quantile rank histogram statistics and plots (t-test and others to be added) # Part SYNOP -1. 'main_synop.sh' - main bash-script to run Apps for observational data for synoptical stations. -2. 'gsv_mod_data.sh' - bash-script to run pre-processing (using cdo's operators) of modelled data extracted with gsv interface over global domain for selected meteorological variables (measured at synoptical station) into hourly time-slices and saving (temporary) to separate hourly nc-files. -3. 'synop_obs.sh' - bash-script to run reading and extraction (using sql with odb_api software) of selected meteorological variables at hourly time-slices for observations from ground-based synoptical stations at fixed at the surface geographical (latitude, longitude) locations over selected geographical domain (or over the globe) and saving (temporary) to dat-file. Note, that list of available meteorological variables in open multi-year data ODB includes: 91 - total amount of clouds; 108 - sea level pressure; 80 - 1-hour precipitation; 58 - relative humidity; 999 - 10-minute precipitation intensity; 71 - snow depth; 39 - 2m temperature; 40 - dew point temperature; 62 - visibility; 111 - wind direction; 261 - maximum wind gust in last 10 minutes; 221 -surface wind speed. -4. 'synop_mod.sh' - bash-script to run reading, extraction and interpolation (using cdo's operators) of modelled data for selected meteorological variables at the same hourly time-slices to geographical (latitude, longitude) locations of synoptical stations, and saving (temporary) to dat-file, and adding (using import with odb_api) such data to ODB. Interpolation is preceeded by constructing unstructured grid based on locations of synop stations and calculating weights for such grid. -5. 'graph_mod_obs.py' - python-script -6. 'synop_stats.sh' - bash-script +1. `main_synop.sh` - main bash-script to run Apps for observational data for synoptical stations. +2. `gsv_mod_data.sh` - bash-script to run pre-processing (using `cdo` operators) of modelled data extracted with gsv interface over global domain for selected meteorological variables (measured at synoptical station) into hourly time-slices and saving (temporary) to separate hourly nc-files. +3. `synop_obs.sh` - bash-script to run reading and extraction (using `sql` with `odb_api` software) of selected meteorological variables at hourly time-slices for observations from ground-based synoptical stations at fixed at the surface geographical (latitude, longitude) locations over selected geographical domain (or over the globe) and saving (temporary) to dat-file. Note, that list of available meteorological variables in open multi-year data ODB includes: 91 - total amount of clouds; 108 - sea level pressure; 80 - 1-hour precipitation; 58 - relative humidity; 999 - 10-minute precipitation intensity; 71 - snow depth; 39 - 2m temperature; 40 - dew point temperature; 62 - visibility; 111 - wind direction; 261 - maximum wind gust in last 10 minutes; 221 -surface wind speed. +4. `synop_mod.sh` - bash-script to run reading, extraction and interpolation (using `cdo` operators) of modelled data for selected meteorological variables at the same hourly time-slices to geographical (latitude, longitude) locations of synoptical stations, and saving (temporary) to dat-file, and adding (using `import` with `odb_api`) such data to ODB. Interpolation is preceeded by constructing unstructured grid based on locations of synop stations and calculating weights for such grid. +5. `graph_mod_obs.py` - python-script to run an internal self-control in calculating differences between observed and modelled data for locations of synoptical stations +6. `synop_stats.sh` - bash-script to run (addtional `bash`, `fortran`, `python` scripts) calculating/producing +- (i) rank histograms for all synop stations for 00, 06, 12 and 18 UTCs (with `produce_rank_histograms_all_stations.sh`, using `rank_histograms_one_station.f95` and `sql/import` with `oadb_api`), +- (ii) standard plots for each synop station (with `produce_standard_plots_all_stations.sh` using `plot_quantiles_rankhist.py` and `sql` with `oadb_api`; see an example of such plot), and +- (iii) summary rank histograms for all stations (`summary_rank_histograms_all_stations.sh` using `rank_histogram_summary_statistics.f95`, `plot_p_values_map.py`, `plot_rank_hist_sum_all_stations.py` and `sql` with `oadb_api`; see examples of such plots). +- Note, needed input includes: list of synop stations with geographical coordinates, modelled data at stations coordinates, pre-computed quantiles as a function of time of year and rank histogram bootstrap mean square deviation (MSD) values. # Disclaimers The OBS package is in a developement phase by the University of Helsinki team, led by Heikki Järvinen (heikki.j.jarvinen@helsinki.fi). The team includes also Jouni Räisänen (jouni.raisanen@helsinki.fi), Lauri Tuppi (lauri.tuppi@helsinki.fi), Madeleine Ekblom (madeleine.ekblom@helsinki.fi), and Alexander Mahura (alexander.mahura@helsinki.fi). Some features are still not implemented and you may expect bugs. For feedback and issue reporting, feel free to open an issue in: https://earth.bsc.es/gitlab/digital-twins/de_340/obs/-/issues