User Tools

Site Tools


tools:ocean_postprocessing

This is an old revision of the document!


Objective

Set of bash functions called through the ocean_pp.bash script allowing the user to perform a series of diagnostics.

Description

The bash script ocean_pp.bash performs diagnostics on monthly or daily ocean model outputs. It relies on a set of functions included in common_ocean_post.txt that can also be used within the Ec-Earth and NEMO post templates when using autosubmit. Many of the functions included in common_ocean_post.txt have been built based on Hui Du's post-processing scripts.

The outputs of ocean_pp.bash are of 3 types :

  • 63 different indexes (sea ice volume, MOC intensity, integrated heat content …) which can all be uploaded in R through CFU_load (see diagnostics section)
  • 19 different vertical sections (meridional overturning streamfunction, temperature sections at a given latitude/longitude) which can not be loaded through CFU_load, but the manual loading can be done directly from the files output by ocean_pp.bash
  • 8 different 2-dimensional (maps of ocean and salt heat content over different layers) which are outputs on the original ORCA grid. Then, those outputs need to be interpolated (see interpolation section) before a manual loading in R.
  • any of the original outputs from the model without any other post-processing than concatenating the forecast times, gathering the members and formatting.

If you post-process daily or monthly means the output directory is daily_mean or monthly_mean.

Some more specifications:

  • The dependency between the diagnostics is handled, i.e. even if you ask for a diagnostic that depends on another diagnostic, you don't have to worry about the order in which you ask the diagnostics in ocean_pp.sh or even to worry about asking the required pre-diagnostics. ocean_pp.sh will handle that automatically. Ex: If you ask for ('max_moc' 'moc'), ocean_pp.sh will process ('moc' 'max_moc'). If you ask for ('area_moc' 'ohc'), ocean_pp.sh will process ('moc' 'area_moc' 'ohc')
  • If you extend the number of members of an experiment or its forecast length, the output will be gathered on cfunas. Ex : You have already postprocessed for exp t001: 1) members 0-2 between leadtimes 1 and 36, 2) members 0 between leadtimes 37 and 60 If you process member 1-2 between leadtimes 37 and 60, you'll find on the nas a single output with members 0-2 between leadtimes 1 and 60 BUT : You have already postprocessed for exp t001: 1) members 0-2 between leadtimes 1 and 36, 2) members 0 between leadtimes 37 and 60 If you process member 2 between leadtimes 25 and 60, you'll find the two above files plus a new one with members 2 between leadtimes 25 and 60 on the nas

ocean_pp.bash handles any range of leadtimes instead of complete years only. However, the max_moc diagnostic can not be computed if you do not have complete years since it is a diagnostic computed from annual means.

ocean_pp.bash also post-processes daily means, instead of only monthly means. You only need to specify as output directory daily_mean instead of monthly_mean because the number of time steps and dates are read from the output. It has been tested with NEMO only simulations but this required some changes in the NEMO templates so that some changes in the EcEarth templates might be needed also before using this option for EcEarth simulations.

List of functions

  • reduce_mmo
  • get_diagsMMO
  • get_nemovar
  • get_glorys
  • clean_diagsMMO
  • vertmeansal
  • heat_sal_mxl
  • ohc_specified_layer
  • moc
  • convection
  • psi
  • gyres
  • area_moc
  • max_moc
  • siasiesiv
  • ohc
  • cutsection
  • interp3d
  • setminmax
  • concat
  • gather_memb

Requirements

This package needs cdftools 2.1, cdo and nco.

Use

To perform the postprocessing of the files, you must run ./ocean_pp.bash with a configuration file (config_file-ocean_pp.bash) available in the git project. Through this configuration file, you can specify which diagnostics you want to run, which variables to extract, the dates and members, etc… Everything is detailed in the configuration file.

  • Sections raw_vars_ocean and raw_vars_ice allow to specify which 2D variables you want to extract:

If “default” option is specified, it will treat the same variables as before (sosstsst, sosaline and somxl010 for ocean, isnowthi, iicethic, ileadfra, iicetemp, ice_pres for sea ice). If nothing is specified, it will extract all the 2D variables present in the input files. Otherwise, it will treat exclusively the variables listed.

A README file describes the various diagnostics which are available as options through the listpost argument.

Repository

The link to the GIT repository is https://gitlab.cfu.local/cfu/ocean_diagnostics.git

Contact

The coordinator of this project is Pierre-Antoine Bretonnière pierre-antoine.bretonniere@ic3.cat. If Pierre-Antoine is not available, you can also ask Virginie Guemas virginie.guemas@ic3.cat in case of bugs.

Previous version of this page can be found here: http://ic3.cat/wikicfu/index.php/Tools/Postprocessing#Ocean_postprocessing

Development

If you want to include a new diagnostic. Look for the @ signs in the common_ocean_post.txt then ocean_pp.bash and you'll find the locations where you have to include lines of code and some additional instructions so that everything works fine. Feel free to ask if something is not clear. To facilitate testing of new functions or of new options in existing functions, you can use the script named setup_development_oriol.bash in any working directory.

On this page (http://ic3.cat/wikicfu/index.php/Ocean_diagnostics_survey), the users can add the requests they have for new diagnostics they think are missing.

If you want to make available the post-processing tools for new grids, you need to choose a flag for this new grid (such as Ec2.3_O1L42), and then prepare all the configuration files such as done here : /cfu/autosubmit/con_files/ for example for any other flags. Namely you need to prepare the files :

  • depth.flag.txt # list of depths of the model, can be obtained by printing the values from the meshmask
  • mesh_mask_nemo.flag.nc # meshmask available with the model release
  • new_maskglo.flag.nc # mask of various basins, it has to be prepared manually from the tmask in the meshmask
  • weights/flag/rmp_flag_to_*_lev[1 to max level].nc # SCRIP weights to interpolate each ocean model level toward a regular grid (usually the corresponding atmospheric model grid but you can choose whichever you want)

How to develop

Download the repository folder to your computer ( git clone https://dev.cfu.local/ocean_diagnostics.git <local_folder> ) and check-out to the branch you want to develop in. You can find GIT help here: http://ic3.cat/wikicfu/images/GIT_introduction.pdf

GIT branching scheme

See the following page to check the current branching scheme used within the GIT project 'Ocean postprocessing': Git branching scheme

Style Guide

You can check the style guide for the ocean_diagnostics package here

tools/ocean_postprocessing.1432639493.txt.gz · Last modified: 2015/10/19 14:55 (external edit)