diff --git a/.gitignore b/.gitignore index 671056823a03468d57fc442ceefaced77bcdd6c9..0e0da37de80b41efcf98df72fd407f2e5221282f 100644 --- a/.gitignore +++ b/.gitignore @@ -3,33 +3,18 @@ *.mod *~ *.log -work_* -sources/oasis3-mct/generated __pycache__ -tests/slurm-* -tests/.coverage* -tests/data/primavera/ -tests/htmlcov -tests/.gitignore -tests/amip.yaml.sh -tests/data/ICMGGa22e+000000 -tests/data/grids-noamip.nc -tests/data/masks-noamip.nc -tests/data/areas-amip.nc -tests/data/bak/ -#tests/data/ece3-toy-model/ -tests/data/grids-amip.nc -tests/data/masks-amip.nc -tests/data/test_word_break.sh -tests/data/tmp1.nc -sources/oasis3-mct/util/make_dir/make.inc -tests/data/ece3-toy-model/areas.nc -tests/data/ece3-toy-model/grids.nc -tests/data/ece3-toy-model/masks.nc -tests/data/ece3-toy-model/rmp_* -tests/data/ece3-toy-model/rstas2.nc -tests/data/ece3-toy-model/rstas.nc -tests/data/ece3-toy-model/rstos.nc -tests/data/areas-noamip.nc + +# data +data/ + +# scripts +scripts/slurm-* + +# sources sources/oasis3-mct/util/make_dir/make.inc +sources/oasis3-mct/generated + + +# docs docs/A_Ice_frac_optimized.gif diff --git a/README.md b/README.md index 825948b87e8e607f8b2f2cab7fa700d021432b87..569bb1516888698395b89400029bde77061e3ee4 100644 --- a/README.md +++ b/README.md @@ -1,268 +1,49 @@ -# Python AMIP reader -Conversion from Fortran to Python of the EC Earth 4 project AMIP reader using the OASIS API from pyOASIS. This repository contains tests for running both implementations (Fortran code is taken from [EC Earth v3.3.3.1](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/tree/3.3.3.1)) using the current setup for IFS and AMIP from EC Earth v3.3.3.1. - -The AMIP reader provides realistic sea surface temperature and sea ice data from 1979 to near present, thus enabling scientists to focus on the atmospheric model without the added complexity of ocean-atmosphere coupled model feedbacks. It is not meant to be used for climate change prediction, an endeavor that requires a coupled atmosphere-ocean model (e.g., see AMIP's sister project CMIP). - -The new Python AMIP reader is a flexible, highly-customizable and ready out-of-the-box component with the following features: -* **Read and send any field** by including it in the [new namelist file](#new-namelist-file) and making sure the input netCDF files follow a [common structure](#add-a-new-field). The following fields have been tested: - * SST (Sea Surface Temperature) and SIC (Sea Ice Concentration) constructed **mid-month fields** (exactly as the AMIP reader from [EC Earth v3.3.3.1](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/tree/3.3.3.1)). - * SST (Sea Surface Temperature) and SIC (Sea Ice Concentration) constructed **daily fields** (exactly as the AMIP reader for PRIMAVERA from [EC Earth v3.2.2 (PRIMAVERA production)](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/tree/3.2.2_Primavera_production)). - * Gridded emissions data as in these [TM5 routines](https://earth.bsc.es/gitlab/svn/ecearth-mirror/-/blob/ceb1f5b95ced1374f190d2a9f9b833a74627c5d8/sources/tm5mp/proj/co2/emission_read__co2.F90) from EC Earth in `m5mp/proj/co2emission_read__co2.F90`. -* **Backwards compatibility** with all the namelist files used by the old Fortran AMIP reader. -* **Dynamic `grids.nc` and `masks.nc` definitions** for OASIS based on the input netCDF files. It will not override any existing grid with the same name (as stated in the [OASIS 4.0 user guide](http://www.cerfacs.fr/oa4web/oasis3-mct_4.0/oasis3mct_UserGuide.pdf)), providing backwards compatibility with all current implementations. - -All the source code for the new implementation can be found under the folder [`sources/amip-forcing/src/python/`](sources/amip-forcing/src/python/) of this repository. - -A report on the implementation can be found in the BSC Earth Wiki at https://earth.bsc.es/wiki/lib/exe/fetch.php?media=degree_students:degree_thesis_rmartin.pdf . - -## Table of contents -- [Python dependencies](#python-dependencies) -- [Usage](#usage) - - [New namelist file](#new-namelist-file) - - [Example of the new namelist file](#example-of-the-new-namelist-file) - - [Backwards compatibility](#backwards-compatibility) - - [Add a new field](#add-a-new-field) - - [Input netCDF files](#input-netcdf-files) - - [Hardcode an operation (inner workings)](#hardcode-an-operation-inner-workings) -- [Run an example](#run-an-example) - - [Download the datasets](#download-the-datasets) - - [Compile](#compile) - - [Run](#run) -- [Run the tests](#run-the-tests) -- [Visualize netCDF output files](#visualize-netcdf-output-files) - - [Generate GIF files](#generate-gif-files) - - -## Python dependencies -The new Python implementation relies on the Python modules `f90nml`, `netCDF4` and `numpy`. In order to install these dependencies in your local machine you may execute the following command. -```bash -pip3 install f90nml netCDF4 numpy -``` -Or if running in MN4 (assuming the modules `impi/2017.4` and `mkl/2017.4` are already loaded): -```bash -module load python/3.6.1 -module load netcdf/4.2 -``` +# EC-Earth Data-Coupler +**NOTE**: Older version of ReadMe is available in [`docs/README_old.md`](./docs/README_old.md) -## Usage -The new Python AMIP reader gets all its configuration from a new `namelist.amip` file. However, it can also be used as **drop-in replacement for the Fortran implementation without any additional configuration** for reading and sending the SST and SIC fields for both EC Earth v3.3.3.1 and EC Earth v3.2.2 (PRIMAVERA production). See the [backwards compatibility section](#backwards-compatibility) for more info. +## Introduction +The Data Coupler is being developed at the BSC to implement a CO2 box model for EC-Earth3 and to read and interpolate CO2 anthropogenic emissions (from e.g. CMIP6 input4MIPS) and CO2 surface fluxes (from e.g. vegetation+fire+ocean fluxes from SiB4) for EC-Earth4-CC. -In order to take advantage of all the new functionalities and have more control over the coupling fields and input data, it is highly recommended to use the [new `namelist.amip` configuration structure](#new-namelist-file). Although not necessary, it is also recommended to avoid setting the AMIP grid in the `grids.nc`, `masks.nc` and `areas.nc` files in advance, as the new AMIP reader is able to write them at runtime. +## Getting the model +The model can be cloned from [`BSC-ES gitlab repository`](https://earth.bsc.es/gitlab/es/python-amip-reader) -### New namelist file -Although the new AMIP reader works with the old `namelist.amip` configuration files, the new namelist configuration file provides greater flexibility to the user when reading and operating with the input data. Here is a scheme for the new namelist generator bash script taken from [`namelist_python.amip.sh`](./tests/data/namelist_python.amip.sh): -```bash -cat << EOF -!----------------------------------------------------------------------- -&NAMAMIP -!----------------------------------------------------------------------- - RunLengthSec = ${leg_length_sec} - TimeStepSec = ${cpl_freq_amip_sec} - StartYear = ${leg_start_date_yyyymmdd:0:4} - StartMonth = ${leg_start_date_yyyymmdd:4:2} - StartDay = ${leg_start_date_yyyymmdd:6:2} - FixYear = ${ifs_cmip_fixyear} - # Vars(1,:) = - # ... - # Vars(n,:) = ... - LDebug = false -!----------------------------------------------------------------------- -/ -EOF -``` -The main difference between the old and the new namelist files is the substitution of the fields exclusively defined for SST and SIC grids (`FileListSST`, `FileListSIC` and `LInterpolate`) for an agnostic array of fields `Vars(:,:)`. `Vars(:,:)` shape is `(n,13)`, being `n` the number of fields to exchange and each coupling field declaration must follow the structure below: - -| Index | Name | Type | Description | -| ------ | ------ | ----------------------- | ------------------------- | -| 0 | id | `string` | Unique identifier for the field. | -| 1 | grid_name | `string` | Grid name (must match one of the grids declared in the `namcouple` file). | -| 2 | oasis_name | `string` | Variable name (must match one of variables under the `grid_name` declared in the `namcouple` file). | -| 3 | file_pattern | `string` | File pattern of the input netCDF files. I.e: `'HadISST2_prelim_0to360_alldays_sst_[year].nc'`. All patterns must be between brackets `'[]'` and currently the only accepted pattern is: `year`. | -| 4 | netcdf_variable | `string` | Variable to read in the netCDF files. | -| 5 | yref_min | `int32` | Reference year for the `time` variable in the netCDF files. | -| 6 | yref_max | `int32` | The last year of the netCDF files. | -| 7 | timestep | `'monthly'` \| `'daily'` | The step in number of days of the `time` variable in the netCDF files. | -| 8 | interpolate | `true` \| `false` | Whether to interpolate or not (disabled if the timestep is set to `daily`). | -| 9 | scale_factor | `float64` | Scale factor applied after reading the input data. | -| 10 | offset | `float64` | Offset applied after reading the input data. | -| 11 | min | `float64` \| `None`* | Minimum value used to clip the input data before submitting the field. | -| 12 | max | `float64` \| `None`* | Maximum value used to clip the input data before submitting the field. | - -\* In order to set a section as `None` you must leave blank the content between its comas. I.e: for setting the last section (`max`) as `None` in AMIP_sst, the field `Vars(1,:)` ends with `..., 273.15, 271.38, ,`. See the [example of the new namelist file](#example-of-the-new-namelist-file) for the complete configuration. - -Below is an example of the old `namelist.amip` file used for AMIP forcing in EC Earth v3.3.3.1 next to the new equivalent one created by the script above: - -#### Example of the old namelist file -``` -!----------------------------------------------------------------------- -&NAMAMIP -!----------------------------------------------------------------------- - RunLengthSec = 5097600 - TimeStepSec = 86400 - StartYear = 1991 - StartMonth = 01 - StartDay = 01 - FixYear = 0 - FileListSST = 'tosbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc' - FileListSIC = 'siconcbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc' - LDebug = false - LInterpolate = true -!----------------------------------------------------------------------- -/ -``` - -#### Example of the new namelist file -``` -!----------------------------------------------------------------------- -&NAMAMIP -!----------------------------------------------------------------------- - RunLengthSec = 5097600 - TimeStepSec = 86400 - StartYear = 1991 - StartMonth = 01 - StartDay = 01 - FixYear = 0 - Vars(1,:) = 'AMIP_sst_monthly', 'AMIP', 'AMIP_sst', 'tosbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc', 'tosbcs', 1870, 2016, 'monthly', true, 1, 273.15, 271.38, , - Vars(2,:) = 'AMIP_sic_monthly', 'AMIP', 'AMIP_sic', 'siconcbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc', 'siconcbcs', 1870, 2016, 'monthly', true, 0.01, 0, 0, 1, - LDebug = false -!----------------------------------------------------------------------- -/ -``` - -#### Backwards compatibility -The new AMIP reader is backwards compatible with both EC Earth v3.3.3.1 and EC Earth v3.2.2 (PRIMAVERA production) namelist files. It achieves this by internally converting the configuration from the old namelist files to the new representation when reading the namelist file (source code in function `read_namelist()` in [`amip_utils.py`](./sources/amip-forcing/src/python/amip_utils.py)). Here is the internal conversion of the SST coupling field when using each old namelist file: - -##### EC Earth v3.3.3.1 (AMIP forcing) -| Property | Value | -| ---------- | ----------------------- | -| id | `'AMIP_sst_monthly'` | -| grid_name | `'AMIP'` | -| oasis_name | `'AMIP_sst'` | -| file_pattern | `FileListSST` content from the old namelist file. | -| netcdf_variable | `'tosbcs'` | -| yref_min | `1870` | -| yref_max | `2016` | The last year of the netCDF files. | -| timestep | `'monthly'` | -| interpolate | `LInterpolate` content from the old namelist file. | -| scale_factor | `1` | -| offset | `273.15` | -| min | `271.38` | -| max | `None` | - -##### EC Earth v3.2.2 (PRIMAVERA production) -| Property | Value | -| ---------- | ----------------------- | -| id | `'AMIP_sst_daily'` | -| grid_name | `'PSST'` | -| oasis_name | `'AMIP_sst'` | -| file_pattern | `AmipFileRoot` content from the old namelist file appending `'_sst_[year].nc'`. | -| netcdf_variable | `'sst'` | -| yref_min | `1850` | -| yref_max | `2016` | The last year of the netCDF files. | -| timestep | `'daily'` | -| interpolate | `False` | -| scale_factor | `1` | -| offset | `0` | -| min | `None` | -| max | `None` | - -### Add a new field -Adding a new field solely consists on adding the input netCDF files to the runtime folder of the program and adding a new entry to the `Vars(:,:)` field in the namelist following the structure mentioned above (see the [new namelist file](#new-namelist-file) section). - -#### Input netCDF files -All input netCDF files must contain the following variables (assuming the dimensions `time`, `lat` and `lon` are defined): - -| Variable | Type | Shape | Description | -| --------- | ------ | ------------------------ | ------------------------- | -| `time` | `double` | `(time)` | Days since the reference year defined as `yref_min` in its namelist entry. | -| `lat` or `latitude` | `double` | `(lat)` | Latitude. | -| `lon` or `longitude` | `double` | `(lon)` | Longitude. | -| Input variable name | `double` or `float` | `(time, ..., lat, lon)` | The variable name must match the `netcdf_variable` value in its namelist entry. | - -### Hardcode an operation (inner workings) -If it is necessary to hardcode an operation with respect to one of the coupling fields, it is highly recommended to edit the code within the [`AMIPVar` class](sources/amip-forcing/src/python/amip_var.py). Right now, there are already hardcoded operations used to match the old Fortran implementation or read an specific index of the input netCDF variable. Here are some examples: - -```python -logging.debug('{}: no time interpolation t,t2 {} {}'.format(self.id, t_local, self.t2)) -# Hardcoded to match Fortran's AMIP clipping avoidance -if self.oasis_name == 'AMIP_sst': - var_min = np.NINF -``` - -```python -# This section can be "hardcoded" by the user -# Hardcoded CO2 emissions data. Read only sector id 1: Energy -if self.var_name == 'CO2_em_anthro': - raw_field = raw_field[1] -``` - -While the Python implementation was originally inspired by the Fortran implementation, it has now been refactored to take advantage of Python's OOP and simpler modules. For a deeper understanding, here are the simplified sequence and class diagrams of the current implementation. - -![Sequence_Diagram](docs/Sequence_Diagram.png) - -![Class_Diagram](docs/Class_Diagram.png) - -## Run an example -Running an example consists on executing one of the implementations (Fortran or Python) along with an IFS tyo model. This toy model simulates IFS reception behavior as designed in EC Earth v3.3.3.1. The detailed specification can be found below, with the coupling interval and run length taken directly from the `namelist.amip` file. -```python -# Constants -NAMELIST_FILE_NAME = 'namelist.amip' -L080_NX = 35718 # number of LAND grid cells at T159 resolution -L128_NX = 88838 # number of LAND grid cells at T255 resolution -L256_NX = 348528 # number of LAND grid cells at T511 resolution -``` - -**IMPORTANT NOTICE: This documentation assumes you are running an UNIX-based OS with Python +3.6 in the PATH, understand the basics of OASIS and are able to compile and run pyOASIS.** - -The bash scripts for running the examples rely on the command `cdo` for checking the results. In order to install this dependency, you may execute the following code. -```bash -apt install -y cdo ``` -Or if running in MN4: -```bash -module load cdo/1.9.3 +git clone https://earth.bsc.es/gitlab/es/python-amip-reader.git +cd python-amip-reader +git checkout data-coupler ``` -### Download the datasets -The easiest way to download the datasets is to copy the datasets to your copy from 2 folders `tests/data/forcing/ and tests/data/co2/ from @etourign located here: +### Setting up the model +First copy the datasets from Dataset repositories based on the machine. -on BSC marenostrum4 : /gpfs/scratch/bsc32/bsc32051/git/python-amip-reader - -on ECMWF HPC2020 : /hpcperm/c3et/work/python-amip-reader +**on MN4 :** +``` +datadir=/gpfs/scratch/bsc32/bsc32013/data-coupler/data +``` +**on HPC2020:** +``` +datadir=/hpcperm/c3ap/data-coupler/data +``` -### Download the datasets (deprecated) +and then copy them as follows: -Due to the large size of some of the required files and datasets, they are not provided in this repository. It is necessary to download them from MN4 before executing any code. Use the following script to download the required files (note you must change `bsc32074` to your username in MN4): -```bash -scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/oasis/AMIP/* ./tests/data/forcing -scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/amip-forcing/* ./tests/data/forcing -# PRIMAVERA dataset -scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.2.1/inidata/oasis/AMIP-reader/OLD_AMIP_NL/* ./tests/data/primavera -scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.2.2/inidata/amip-primavera/* ./tests/data/primavera -# Emissions dataset -scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/trunk/inidata/tm5/CMIP6/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412.nc ./tests/data/co2 -``` -If you are downloading the data from PRIMAVERA (+74 GB), it is highly recommended to do it in the background: -```bash -nohup scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.2.2/inidata/amip-primavera/* ./tests/data/primavera > nohup.out 2>&1 -# ctrl + z -bg ``` - -Or if running in MN4: -```bash -cp /gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/oasis/AMIP/* ./tests/data/forcing -cp /gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/amip-forcing/* ./tests/data/forcing -cp /gpfs/projects/bsc32/models/ecearth/v3.2.1/inidata/oasis/AMIP-reader/OLD_AMIP_NL/* ./tests/data/primavera -cp /gpfs/projects/bsc32/models/ecearth/v3.2.2/inidata/amip-primavera/* ./tests/data/primavera -cp /gpfs/projects/bsc32/models/ecearth/trunk/inidata/tm5/CMIP6/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412.nc ./tests/data/co2 +cd python-amip-reader +mkdir data +cp -r $datadir/* data/ +``` +or, if you don't want to duplicate data and will not add any new dataset, create a link to the main Dataset folder. +``` +cd python-amip-reader +ln -s $datadir data ``` -### Compile +## Compile Each of the example codes is located in its respect folder inside the [`./sources/amip-forcing/`](./sources/amip-forcing/) folder ([`src/fortran/`](./sources/amip-forcing/src/fortran/) and [`src/python/`](./sources/amip-forcing/src/python/)). In the source folder for the Fortran code there is a [Makefile](./sources/amip-forcing/src/fortran/Makefile). Before compiling OASIS, you must create a `make.inc` file which redirects to a make file with your configuration. You may issue the following command to create your own `make.inc` file: + ```bash cp sources/oasis3-mct/util/make_dir/make.templ.inc sources/oasis3-mct/util/make_dir/make.inc ``` @@ -285,21 +66,48 @@ curr_path = $(dir $(realpath $(lastword $(MAKEFILE_LIST)))) include $(curr_path)/make.hpc2020 ``` -**WARNING: `make.mn4` and `make.hpc2020` assume the proper modules are loaded.** +**WARNING**: `make.mn4` and `make.hpc2020` assume the proper modules are loaded. -For make.mn4 this is done with the following command: `module load python/3.6.1 netcdf/4.2` +For `make.mn4` this is done with the following command: -For make.hpc2020 this is done with `module reset ; module load python3/3.10.10-01 prgenv/intel intel/2021.4.0 hpcx-openmpi/2.9.0 hdf5-parallel/1.12.2 netcdf4-parallel/4.9.1` +``` +module load impi/2017.4 +module load mkl/2017.4 +module load intel/2021.4 +module load python/3.6.1 +module load netcdf/4.2 +module load cdo/1.9.3 +``` -For other platforms you may need to modify or add a new makefile for your specific MPI setup. +For `make.hpc2020` the following: -By executing the following command, you will compile OASIS, pyOASIS and the Fortran implementation of this model. -```bash -(cd ./sources/amip-forcing/src/fortran && make clean) -(cd ./sources/amip-forcing/src/fortran && make oasis) -(cd ./sources/amip-forcing/src/fortran && make) ``` +module reset ; +module load python3/3.10.10-01 +module load prgenv/intel +module load intel/2021.4.0 +module load hpcx-openmpi/2.9.0 +module load hdf5-parallel/1.12.2 +module load netcdf4-parallel/4.9.1 +``` + + +For **MN4** and **HPC2020** you can use the following snippet to build oasis +``` +cd $HPCPERM/work/python-amip-reader/sources/oasis3-mct/util/make_dir +cp make.templ.inc make.inc +# change this line +# include $(curr_path)/make.mn4 +# to +# include $(curr_path)/make.hpc2020 + +cd $HPCPERM/work/python-amip-reader/sources/amip-forcing/src/fortran + +module reset ; module load python3/3.10.10-01 prgenv/intel intel/2021.4.0 hpcx-openmpi/2.9.0 hdf5-parallel/1.12.2 netcdf4-parallel/4.9.1 + +make realclean && make oasis && make +``` If this is the first time you are compiling pyOASIS in the current session, you must update `LD_LIBRARY_PATH` and `PYTHONPATH` environment variables. Luckily, pyOASIS provides a automatic script: ```bash source sources/oasis3-mct/generated/python/init.sh @@ -309,74 +117,126 @@ source sources/oasis3-mct/generated/python/init.csh ``` Alternatively, these lines could be added to your `~/.bashrc` or `~/.cshrc` file. -### Run -In order to run the example code with your desired programming language you must execute the script `run_example.sh` inside the [`tests/`](./tests/) folder. Here is an example on how to run the example with the AMIP reader running in Python and the IFS toy model with 4 MPI processes from 1990-01-01 to 1991-01-01 without a fixed year. -```bash -bash ./run_example.sh python forcing 4 1990-01-01 1991-01-01 0 L128 true -``` +For other platforms you may need to modify or add a new makefile for your specific MPI setup.For more information lease check the [`README_old.md`](./docs/README_old.md#compile) -Here is the complete specification for the script: -```bash -./run_example.sh -``` +## Running the model -Running this script will create a new folder in the root folder of this repository called `work________` with all the output and debug information from the run. +To run the model you need to edit file  `$HPCPERM/work/python-amip-reader/scripts/launch-hpc2020.cmd` and make sure the proper configuration is un-commented at the end, for example -## Run the tests -Execute the following command in the root folder of this repository to run all tests **except the large test** (make sure to meet all the [requirements](#requirements) stated above). -If you are on MN4, it is highly recommended to start first an interactive session: +For a simple *AMIP* run using the amip-forcing : ``` -salloc -p interactive +bash ./run_example.sh datacoupler forcing 4 2005-01-01 2005-03-01 0 128 true false template_conf_datacoupler_forcing.yaml ``` - -```bash -bash run_tests.sh +For the *AMIP Forcing + co2flux* model : ``` -If you also want to run the large test along with the other ones, use the following: -```bash -bash run_tests.sh 1 +bash ./run_example.sh datacoupler co2flux 4 2005-01-01 2005-03-01 0 128 true false template_conf_datacoupler_co2flux.yaml +``` +For the *CO2BOX* model : +``` +bash ./run_example.sh datacoupler co2box 1 2005-01-01 2005-03-01 0 128 true false template_conf_datacoupler_co2box.yaml ``` -This command will run both implementations (Fortran and Python) of the AMIP reader and check if all of them result in the same output. Here is an example of a successful test run: +### YAML input structure + +1. **DataCoupler** + +Three YAML templates are available for DataCoupler variations: +- [`template_conf_datacoupler_forcing.yaml`](./scripts/template_conf_datacoupler_forcing.yaml) a replacement of the EC-Earth3 amip-forcing which reads SST and SIC from input4MIPS and sends them to IFS +- [`template_conf_datacoupler_co2box.yaml`](./scripts/template_conf_datacoupler_co2box.yaml) : a global CO2 box model which reads CEDS CO2 surface concentrations and emissions from input4MIPS and sends a global, surface CO2 field to IFS +- [`template_conf_datacoupler_co2flux.yaml`](./scripts/template_conf_datacoupler_co2flux.yaml) : a basic CO2 flux coupler which reads CEDS CO2 emissions from input4MIPS and sends dummy land,ocean and anthropogenic CO2 fluxes to IFS + +They include 3 sections *Model Information*, *Run Information* and *Coupling Information*, some of which should be filled by the users and some (mainly the *Run Information*) are filled by the [`run_example.sh`](./scripts/run_example.sh) script as it is executed. The generated YAML input will be stored under `name DataCoupler_exp_conf.yaml` . The *Coupling Information* section contains a list of files to read (`FileInputVars`, containing information on the netcdf file and variable to read and other details), and another list of oasis exchange fields between the Data Coupler and the IFS toy model (`OasisOutputVars`, containing details on the sending and receiving of the fields). At every timestep of duration `TimeStepSec` each FileInputVar is read and its content sent to the OasisOutputVar corresponding to its oasis_name field. For the given example below, the `AMIP_sst_monthly` variable is sent by the DataCoupler as the `AMIP_sst` oasis field, which is received by the IFS toy model as the `A_SST` field. + +Below, you will find a copy of the [`template_conf_datacoupler_forcing`](./scripts/template_conf_datacoupler_forcing.yaml) config file. + ``` -Running tests WITHOUT the large test. Use ./run_tests.sh 1 to run all tests including the large test. -***************************************************************** -Running tests... -***************************************************************** -Checking results... -Checking ./work_python_forcing_2_1892-01-01_1894-01-01_0_L128_true vs ./work_fortran_forcing_2_1892-01-01_1894-01-01_0_L128_true -cdo diff: Processed 2 variables over 1462 timesteps [0.23s 52MB]. -cdo diff: Processed 2 variables over 1462 timesteps [0.22s 52MB]. -cdo diff: Processed 2 variables over 1462 timesteps [0.30s 52MB]. -cdo diff: Processed 2 variables over 1462 timesteps [0.31s 52MB]. -... -Tests PASSED +cat << EOF +# YAML input file for DataCoupler, It will be ingested and turn to Experiment YAML file + +## Model Information +ModelNameSend : AMIPFORC +ModelNameReceive : IFS_TOY +LogFileName : amip.log + +## Run Information +RunLengthSec : ${leg_length_sec} +TimeStepSec : ${cpl_freq_amip_sec} +StartYear : ${leg_start_date_yyyymmdd:0:4} +StartMonth : ${leg_start_date_yyyymmdd:4:2} +StartDay : ${leg_start_date_yyyymmdd:6:2} +FixYear : ${ifs_cmip_fixyear} +GridInfo : 88838 # number of grid cells : L080_NX = 35718 @ T159 resolution / L128_NX = 88838 @ T255 resolution / L256_NX = 348528 @ T511 resolution + +## Coupling Information +FileInputVars: + AMIP_sst_monthly : { id : AMIP_sst_monthly, grid_name: AMIP, oasis_name: AMIP_sst, file_pattern: tosbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc, netcdf_variable: tosbcs, yref_min : 1870, yref_max: 2016, timestep: monthly, interpolate : true, scale_factor: 1, offset: 273.15, min: 271.38, max: , update: true, accum: true} + AMIP_sic_monthly : { id : AMIP_sic_monthly, grid_name: AMIP, oasis_name: AMIP_sic, file_pattern: siconcbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc, netcdf_variable: siconcbcs, yref_min : 1870, yref_max: 2016, timestep: monthly, interpolate : true, scale_factor: 0.01, offset: 0, min: 0, max: 1, update: true, accum: true} + +OasisOutputVars: + AMIP_sst : { send_id: AMIP_sst, receive_id: A_SST, send_grid_name: 'AMIP', receive_grid_name: L128, timestep: 'daily', scale_factor: 1, offset: 0, reset: true } + AMIP_sic : { send_id: AMIP_sic, receive_id: A_Ice_frac, send_grid_name: 'AMIP', receive_grid_name: L128, timestep: 'daily', scale_factor: 1, offset: 0, reset: true } + +LDebug : false +EOF ``` -## Visualize netCDF output files -In order to visualize the results from an experiment, you may first fix the grid and time axis of the `EXPOUT` generated files. First, move into the work directory created when running the experiment. For example: -```bash -cd ./tests/work_python_forcing_4_1990-01-01_1991-01-01_0_L128_true/ +2. **Toy Model** + +Three YAML templates are available for Toy Model scripts, which are used for the EC-Earth3 Toy Model: +[`ifs_toy_model.yaml`](./sources/ece3-toy-model/ifs_toy_model.yaml) +[`nemo_toy_model.yaml`](./sources/ece3-toy-model/nemo_toy_model.yaml) +[`runoff_toy_model.yaml`](./sources/ece3-toy-model/runoff_toy_model.yaml) + +They include information about the *model*, *simulation* and *coupling* for three components of the EC-Earth3 Toy model (IFS, NEMO and Run off), some which should be filled by the users. + +Below, you will find a copy of the [`ifs_toy_model.yaml`](./sources/ece3-toy-model/ifs_toy_model.yaml). + + ``` +# yaml metadata -Then, from inside that directory, run the following script with the start date as the argument: -```bash -bash ./convert-ece4pyreader-grids.sh 19900101 +model: + ModelName : IFS_TOY + +simulation: + coupling_interval : 2700 + run_length_sec : 86400 + # Definition of the local partition grid cell + # L080_NX = 35718 # number of LAND grid cells at T159 resolution + # L128_NX = 88838 # number of LAND grid cells at T255 resolution + # L256_NX = 348528 # number of LAND grid cells at T511 resolution + nx : 88838 + +coupling: + out_vars : [A_TauX_oce, A_TauY_oce, A_TauX_ice, A_TauY_ice, A_Qns_mix, A_Qs_mix, A_Evap_total, A_Precip_liquid, A_Precip_solid, A_Qs_ice, A_Qns_ice, A_dQns_dT, A_Evap_ice, A_Runoff, A_Calving] + in_vars : [A_SST, A_Ice_temp, A_Ice_albedo, A_Ice_frac, A_Ice_thickness, A_Snow_thickness] + restart_file : /path/ ``` -As a result, you will now have 8 new files in the working directory for the experiment: `A_Ice_frac.nc`, `A_Ice_frac.grb`, `A_SST.nc`, `A_SST.grb`, `AMIP_sst.nc`, `AMIP_sic.nc`, `areas_AMIP.nc`, `areas_L128.grb`, `areas_L128.nc`, `masks_AMIP.nc`, `masks_L128.grb` and `masks_L128.nc`. +### Submitting runs -### Generate GIF files -Optionally, you may want to save visualization as a GIF file. In order to achieve this, you may first run `./convert-ece4pyreader-grids.sh` as before and then the script `nc_to_gif.py` located in the [`tests/`](./tests/) folder. If you are in the working folder and want to visualize the `A_Ice_frac.nc` file, you may run the following command: +There are 2 ways to run the model:  -```bash -./nc_to_gif.py work_python_forcing_4_1990-01-01_1991-01-01_0_L128_true A_Ice_frac +1. run an **interactive session** and launch the job with bash (for short runs and debugging) + +``` +ecinteractive -c6 -t 12:00:00 -m 32GB +cd $HPCPERM/work/python-amip-reader/scripts/ +bash launch_ece3_data_coupler_hpc2020.cmd ``` -Here is the complete specification for the script: -```bash -./nc_to_gif.py +2. launch the job with **sbatch** (for longer runs) + +``` +cd $HPCPERM/work/python-amip-reader/scripts/ +sbatch launch_ece3_data_coupler_hpc2020.cmd ``` -Running the script will generate two files: a raw GIF file and an optimized one. +If the run is successful, output will be present in a folder representing the run you made e.g. `work_fortran_forcing_4_2005-01-01_2005-03-01_0_128_true` + +Most of the debug output files produced by oasis (containing the fields sent by the data-coupler and received by the ifs toy model) are post-processed to netcdf files which proper grid and time metadata using [`convert-ece4pyreader-grids.sh`](./scripts/convert-ece4pyreader-grids.sh). These files can be viewed with ncview and consulted with cdo or ncdump. + + +## Visualize netCDF output files +Please refer to [`README_old.md`](./docs/README_old.md#visualize-netcdf-output-files) for more information about how to visualize netCDF output files. diff --git a/data.md5 b/data.md5 new file mode 100644 index 0000000000000000000000000000000000000000..6082204995d993381fe443793e9adc5c9a519ca4 --- /dev/null +++ b/data.md5 @@ -0,0 +1,48 @@ +033f343ce3a67f8bb501351f99a85317 data/areas-noamip.nc +e20bd335d965c010b675f20f81e0d46e data/grid-co2_emis.txt +3bbb8b30e6f607153b1f692f75f4c0e1 data/co2/CO2-em-AIR-anthro_input4MIPs_emissions_CMIP_CEDS-2017-08-30_gn_200001-201412.nc +2944ea34fc24351e917c4da477c954b0 data/co2/rmp_CO2_emis_to_B128_GAUSWGT_9.nc +bf613badddb00b8bb7a8ea4c701f54a7 data/co2/masks.nc +63bf264ef75413ebb365218130fd00c2 data/co2/mole-fraction-of-carbon-dioxide-in-air_input4MIPs_GHGConcentrations_CMIP_UoM-CMIP-1-2-0_gr1-GMNHSH_0000-2014.nc +5f57096c16cc1a46e04e43a32c73c8ce data/co2/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412-r1x2.nc +7f8ee4bf120727cfff1045aa1fc9533c data/co2/grids.nc +093662070c6903f56b82ce5240bf67cf data/co2/areas.nc +c0fd8ffdd070907b2c969c8bf36f0d25 data/co2/mole-fraction-of-carbon-dioxide-in-air_input4MIPs_GHGConcentrations_CMIP_UoM-CMIP-1-2-0_gr1-GMNHSH_184901-201412.nc +2c57c30f97f9540f6ea8f7d0f7384ed6 data/co2/co2-area.nc +11b7cf6371d0de75bad15aed846b03e8 data/co2/mole-fraction-of-carbon-dioxide-in-air_input4MIPs_GHGConcentrations_CMIP_UoM-CMIP-1-2-0_gr1-GMNHSH_184901-201412.nc.mod +a6410320ce2ec894d788c93ef07a4e99 data/co2/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412-global.nc.1 +270e2062aa3fa40014c2a79f9e46dd13 data/co2/rmp_AMIP_to_L128_GAUSWGT_9.nc +c0fd8ffdd070907b2c969c8bf36f0d25 data/co2/mole-fraction-of-carbon-dioxide-in-air_input4MIPs_GHGConcentrations_CMIP_UoM-CMIP-1-2-0_gr1-GMNHSH_184901-201412.nc.bak +89f40993d927d188692df6fb621450bf data/co2/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412-global.nc +2a6ace582904da9db74c0de7d9fbfbeb data/co2/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412.nc +90a2c7aac4c2084f7d665bae3ccb842f data/ece3-toy-model/rmp_O1t0_to_L128_GAUSWGT_9.nc +433d647d7a5e963ccf63797ee2fbf911 data/ece3-toy-model/masks.nc +234ac42fa5d5cd076c2e7f4ab37b577b data/ece3-toy-model/rmp_A128_to_O1t0_GAUSWGT_9.nc +b4cf799ea4e4a141d127cc033572648d data/ece3-toy-model/grids.nc +76877af296ecf92f6a5115d2024d9278 data/ece3-toy-model/areas.nc +670f63a39e8a56b7eec8a919f478d5d2 data/ece3-toy-model/rmp_R128_to_RnfA_GAUSWGT_9.nc +02645b463f3a65a929c408c3b8c478e9 data/ece3-toy-model/rmp_A128_to_O1v0_GAUSWGT_9.nc +f8d27ddaa19cf744ab21e313a24c41e4 data/ece3-toy-model/grid-runoff.txt +16dbe2fa3832ff2927117d05b80ad593 data/ece3-toy-model/rmp_RnfO_to_O1t0_GAUSWGT_9.nc +8324c68639fb9547509a92674fdeebcb data/ece3-toy-model/rmp_A128_to_O1u0_GAUSWGT_9.nc +c2841c4f4097f1e8222ff06ddc22cb8d data/ece3-toy-model/rstas.nc +0eacc528218b23b7cfc7f88f05e7d474 data/ece3-toy-model/rstos.nc +f8f32e94f5844e28dcac542d3037f7c7 data/grid-amip.txt +2b7ad9915a0ea47be23d4f2499ac8a39 data/ICMGGa22e+000000 +eb2ddf1dd805be45c83da44316292843 data/forcing/AMIP/masks.nc +0715a0b90cab18235a17b48f1dcfee8b data/forcing/AMIP/grids.nc +3b0a69e73f3c655745df83b9e6763365 data/forcing/AMIP/areas.nc +2b01f60712a7de76ddad74edf0ca2fcc data/forcing/noamip/areas-noamip.nc +962d938652e10c82cf2c4fefb9875039 data/forcing/noamip/masks-noamip.nc +7763084bc6027f84f1d13b04c3b76ef1 data/forcing/noamip/grids-noamip.nc +c083451e0eb8a591b65b8d32c7ef9284 data/forcing/siconcbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc +eb2ddf1dd805be45c83da44316292843 data/forcing/masks.nc +080e5320ad6cdf2d29710d3091376dac data/forcing/amip-area.nc +0715a0b90cab18235a17b48f1dcfee8b data/forcing/grids.nc +3b0a69e73f3c655745df83b9e6763365 data/forcing/areas.nc +370bcbfaec69f457ec53401fba125e27 data/forcing/tosbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc +270e2062aa3fa40014c2a79f9e46dd13 data/forcing/rmp_AMIP_to_L128_GAUSWGT_9.nc +95ce78c393b419557a04906554700ecc data/forcing/tosbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-0_gs1x1_187001-201512.nc +9435d96e3e2f55caa762defbaf630e71 data/forcing/siconcbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-0_gs1x1_187001-201512.nc +f925f778f5cb615cef0856ea4797389a data/masks-noamip.nc +ac6a81c53dd07dfe45c5b399a66de34c data/grids-noamip.nc diff --git a/docs/README_old.md b/docs/README_old.md new file mode 100644 index 0000000000000000000000000000000000000000..825948b87e8e607f8b2f2cab7fa700d021432b87 --- /dev/null +++ b/docs/README_old.md @@ -0,0 +1,382 @@ +# Python AMIP reader +Conversion from Fortran to Python of the EC Earth 4 project AMIP reader using the OASIS API from pyOASIS. This repository contains tests for running both implementations (Fortran code is taken from [EC Earth v3.3.3.1](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/tree/3.3.3.1)) using the current setup for IFS and AMIP from EC Earth v3.3.3.1. + +The AMIP reader provides realistic sea surface temperature and sea ice data from 1979 to near present, thus enabling scientists to focus on the atmospheric model without the added complexity of ocean-atmosphere coupled model feedbacks. It is not meant to be used for climate change prediction, an endeavor that requires a coupled atmosphere-ocean model (e.g., see AMIP's sister project CMIP). + +The new Python AMIP reader is a flexible, highly-customizable and ready out-of-the-box component with the following features: +* **Read and send any field** by including it in the [new namelist file](#new-namelist-file) and making sure the input netCDF files follow a [common structure](#add-a-new-field). The following fields have been tested: + * SST (Sea Surface Temperature) and SIC (Sea Ice Concentration) constructed **mid-month fields** (exactly as the AMIP reader from [EC Earth v3.3.3.1](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/tree/3.3.3.1)). + * SST (Sea Surface Temperature) and SIC (Sea Ice Concentration) constructed **daily fields** (exactly as the AMIP reader for PRIMAVERA from [EC Earth v3.2.2 (PRIMAVERA production)](https://earth.bsc.es/gitlab/es/auto-ecearth3/-/tree/3.2.2_Primavera_production)). + * Gridded emissions data as in these [TM5 routines](https://earth.bsc.es/gitlab/svn/ecearth-mirror/-/blob/ceb1f5b95ced1374f190d2a9f9b833a74627c5d8/sources/tm5mp/proj/co2/emission_read__co2.F90) from EC Earth in `m5mp/proj/co2emission_read__co2.F90`. +* **Backwards compatibility** with all the namelist files used by the old Fortran AMIP reader. +* **Dynamic `grids.nc` and `masks.nc` definitions** for OASIS based on the input netCDF files. It will not override any existing grid with the same name (as stated in the [OASIS 4.0 user guide](http://www.cerfacs.fr/oa4web/oasis3-mct_4.0/oasis3mct_UserGuide.pdf)), providing backwards compatibility with all current implementations. + +All the source code for the new implementation can be found under the folder [`sources/amip-forcing/src/python/`](sources/amip-forcing/src/python/) of this repository. + +A report on the implementation can be found in the BSC Earth Wiki at https://earth.bsc.es/wiki/lib/exe/fetch.php?media=degree_students:degree_thesis_rmartin.pdf . + +## Table of contents +- [Python dependencies](#python-dependencies) +- [Usage](#usage) + - [New namelist file](#new-namelist-file) + - [Example of the new namelist file](#example-of-the-new-namelist-file) + - [Backwards compatibility](#backwards-compatibility) + - [Add a new field](#add-a-new-field) + - [Input netCDF files](#input-netcdf-files) + - [Hardcode an operation (inner workings)](#hardcode-an-operation-inner-workings) +- [Run an example](#run-an-example) + - [Download the datasets](#download-the-datasets) + - [Compile](#compile) + - [Run](#run) +- [Run the tests](#run-the-tests) +- [Visualize netCDF output files](#visualize-netcdf-output-files) + - [Generate GIF files](#generate-gif-files) + + +## Python dependencies +The new Python implementation relies on the Python modules `f90nml`, `netCDF4` and `numpy`. In order to install these dependencies in your local machine you may execute the following command. +```bash +pip3 install f90nml netCDF4 numpy +``` +Or if running in MN4 (assuming the modules `impi/2017.4` and `mkl/2017.4` are already loaded): +```bash +module load python/3.6.1 +module load netcdf/4.2 +``` + +## Usage +The new Python AMIP reader gets all its configuration from a new `namelist.amip` file. However, it can also be used as **drop-in replacement for the Fortran implementation without any additional configuration** for reading and sending the SST and SIC fields for both EC Earth v3.3.3.1 and EC Earth v3.2.2 (PRIMAVERA production). See the [backwards compatibility section](#backwards-compatibility) for more info. + +In order to take advantage of all the new functionalities and have more control over the coupling fields and input data, it is highly recommended to use the [new `namelist.amip` configuration structure](#new-namelist-file). Although not necessary, it is also recommended to avoid setting the AMIP grid in the `grids.nc`, `masks.nc` and `areas.nc` files in advance, as the new AMIP reader is able to write them at runtime. + +### New namelist file +Although the new AMIP reader works with the old `namelist.amip` configuration files, the new namelist configuration file provides greater flexibility to the user when reading and operating with the input data. Here is a scheme for the new namelist generator bash script taken from [`namelist_python.amip.sh`](./tests/data/namelist_python.amip.sh): +```bash +cat << EOF +!----------------------------------------------------------------------- +&NAMAMIP +!----------------------------------------------------------------------- + RunLengthSec = ${leg_length_sec} + TimeStepSec = ${cpl_freq_amip_sec} + StartYear = ${leg_start_date_yyyymmdd:0:4} + StartMonth = ${leg_start_date_yyyymmdd:4:2} + StartDay = ${leg_start_date_yyyymmdd:6:2} + FixYear = ${ifs_cmip_fixyear} + # Vars(1,:) = + # ... + # Vars(n,:) = ... + LDebug = false +!----------------------------------------------------------------------- +/ +EOF +``` + +The main difference between the old and the new namelist files is the substitution of the fields exclusively defined for SST and SIC grids (`FileListSST`, `FileListSIC` and `LInterpolate`) for an agnostic array of fields `Vars(:,:)`. `Vars(:,:)` shape is `(n,13)`, being `n` the number of fields to exchange and each coupling field declaration must follow the structure below: + +| Index | Name | Type | Description | +| ------ | ------ | ----------------------- | ------------------------- | +| 0 | id | `string` | Unique identifier for the field. | +| 1 | grid_name | `string` | Grid name (must match one of the grids declared in the `namcouple` file). | +| 2 | oasis_name | `string` | Variable name (must match one of variables under the `grid_name` declared in the `namcouple` file). | +| 3 | file_pattern | `string` | File pattern of the input netCDF files. I.e: `'HadISST2_prelim_0to360_alldays_sst_[year].nc'`. All patterns must be between brackets `'[]'` and currently the only accepted pattern is: `year`. | +| 4 | netcdf_variable | `string` | Variable to read in the netCDF files. | +| 5 | yref_min | `int32` | Reference year for the `time` variable in the netCDF files. | +| 6 | yref_max | `int32` | The last year of the netCDF files. | +| 7 | timestep | `'monthly'` \| `'daily'` | The step in number of days of the `time` variable in the netCDF files. | +| 8 | interpolate | `true` \| `false` | Whether to interpolate or not (disabled if the timestep is set to `daily`). | +| 9 | scale_factor | `float64` | Scale factor applied after reading the input data. | +| 10 | offset | `float64` | Offset applied after reading the input data. | +| 11 | min | `float64` \| `None`* | Minimum value used to clip the input data before submitting the field. | +| 12 | max | `float64` \| `None`* | Maximum value used to clip the input data before submitting the field. | + +\* In order to set a section as `None` you must leave blank the content between its comas. I.e: for setting the last section (`max`) as `None` in AMIP_sst, the field `Vars(1,:)` ends with `..., 273.15, 271.38, ,`. See the [example of the new namelist file](#example-of-the-new-namelist-file) for the complete configuration. + +Below is an example of the old `namelist.amip` file used for AMIP forcing in EC Earth v3.3.3.1 next to the new equivalent one created by the script above: + +#### Example of the old namelist file +``` +!----------------------------------------------------------------------- +&NAMAMIP +!----------------------------------------------------------------------- + RunLengthSec = 5097600 + TimeStepSec = 86400 + StartYear = 1991 + StartMonth = 01 + StartDay = 01 + FixYear = 0 + FileListSST = 'tosbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc' + FileListSIC = 'siconcbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc' + LDebug = false + LInterpolate = true +!----------------------------------------------------------------------- +/ +``` + +#### Example of the new namelist file +``` +!----------------------------------------------------------------------- +&NAMAMIP +!----------------------------------------------------------------------- + RunLengthSec = 5097600 + TimeStepSec = 86400 + StartYear = 1991 + StartMonth = 01 + StartDay = 01 + FixYear = 0 + Vars(1,:) = 'AMIP_sst_monthly', 'AMIP', 'AMIP_sst', 'tosbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc', 'tosbcs', 1870, 2016, 'monthly', true, 1, 273.15, 271.38, , + Vars(2,:) = 'AMIP_sic_monthly', 'AMIP', 'AMIP_sic', 'siconcbcs_input4MIPs_SSTsAndSeaIce_CMIP_PCMDI-AMIP-1-1-3_gn_187001-201706.nc', 'siconcbcs', 1870, 2016, 'monthly', true, 0.01, 0, 0, 1, + LDebug = false +!----------------------------------------------------------------------- +/ +``` + +#### Backwards compatibility +The new AMIP reader is backwards compatible with both EC Earth v3.3.3.1 and EC Earth v3.2.2 (PRIMAVERA production) namelist files. It achieves this by internally converting the configuration from the old namelist files to the new representation when reading the namelist file (source code in function `read_namelist()` in [`amip_utils.py`](./sources/amip-forcing/src/python/amip_utils.py)). Here is the internal conversion of the SST coupling field when using each old namelist file: + +##### EC Earth v3.3.3.1 (AMIP forcing) +| Property | Value | +| ---------- | ----------------------- | +| id | `'AMIP_sst_monthly'` | +| grid_name | `'AMIP'` | +| oasis_name | `'AMIP_sst'` | +| file_pattern | `FileListSST` content from the old namelist file. | +| netcdf_variable | `'tosbcs'` | +| yref_min | `1870` | +| yref_max | `2016` | The last year of the netCDF files. | +| timestep | `'monthly'` | +| interpolate | `LInterpolate` content from the old namelist file. | +| scale_factor | `1` | +| offset | `273.15` | +| min | `271.38` | +| max | `None` | + +##### EC Earth v3.2.2 (PRIMAVERA production) +| Property | Value | +| ---------- | ----------------------- | +| id | `'AMIP_sst_daily'` | +| grid_name | `'PSST'` | +| oasis_name | `'AMIP_sst'` | +| file_pattern | `AmipFileRoot` content from the old namelist file appending `'_sst_[year].nc'`. | +| netcdf_variable | `'sst'` | +| yref_min | `1850` | +| yref_max | `2016` | The last year of the netCDF files. | +| timestep | `'daily'` | +| interpolate | `False` | +| scale_factor | `1` | +| offset | `0` | +| min | `None` | +| max | `None` | + +### Add a new field +Adding a new field solely consists on adding the input netCDF files to the runtime folder of the program and adding a new entry to the `Vars(:,:)` field in the namelist following the structure mentioned above (see the [new namelist file](#new-namelist-file) section). + +#### Input netCDF files +All input netCDF files must contain the following variables (assuming the dimensions `time`, `lat` and `lon` are defined): + +| Variable | Type | Shape | Description | +| --------- | ------ | ------------------------ | ------------------------- | +| `time` | `double` | `(time)` | Days since the reference year defined as `yref_min` in its namelist entry. | +| `lat` or `latitude` | `double` | `(lat)` | Latitude. | +| `lon` or `longitude` | `double` | `(lon)` | Longitude. | +| Input variable name | `double` or `float` | `(time, ..., lat, lon)` | The variable name must match the `netcdf_variable` value in its namelist entry. | + +### Hardcode an operation (inner workings) +If it is necessary to hardcode an operation with respect to one of the coupling fields, it is highly recommended to edit the code within the [`AMIPVar` class](sources/amip-forcing/src/python/amip_var.py). Right now, there are already hardcoded operations used to match the old Fortran implementation or read an specific index of the input netCDF variable. Here are some examples: + +```python +logging.debug('{}: no time interpolation t,t2 {} {}'.format(self.id, t_local, self.t2)) +# Hardcoded to match Fortran's AMIP clipping avoidance +if self.oasis_name == 'AMIP_sst': + var_min = np.NINF +``` + +```python +# This section can be "hardcoded" by the user +# Hardcoded CO2 emissions data. Read only sector id 1: Energy +if self.var_name == 'CO2_em_anthro': + raw_field = raw_field[1] +``` + +While the Python implementation was originally inspired by the Fortran implementation, it has now been refactored to take advantage of Python's OOP and simpler modules. For a deeper understanding, here are the simplified sequence and class diagrams of the current implementation. + +![Sequence_Diagram](docs/Sequence_Diagram.png) + +![Class_Diagram](docs/Class_Diagram.png) + +## Run an example +Running an example consists on executing one of the implementations (Fortran or Python) along with an IFS tyo model. This toy model simulates IFS reception behavior as designed in EC Earth v3.3.3.1. The detailed specification can be found below, with the coupling interval and run length taken directly from the `namelist.amip` file. +```python +# Constants +NAMELIST_FILE_NAME = 'namelist.amip' +L080_NX = 35718 # number of LAND grid cells at T159 resolution +L128_NX = 88838 # number of LAND grid cells at T255 resolution +L256_NX = 348528 # number of LAND grid cells at T511 resolution +``` + +**IMPORTANT NOTICE: This documentation assumes you are running an UNIX-based OS with Python +3.6 in the PATH, understand the basics of OASIS and are able to compile and run pyOASIS.** + +The bash scripts for running the examples rely on the command `cdo` for checking the results. In order to install this dependency, you may execute the following code. +```bash +apt install -y cdo +``` +Or if running in MN4: +```bash +module load cdo/1.9.3 +``` + +### Download the datasets + +The easiest way to download the datasets is to copy the datasets to your copy from 2 folders `tests/data/forcing/ and tests/data/co2/ from @etourign located here: + +on BSC marenostrum4 : /gpfs/scratch/bsc32/bsc32051/git/python-amip-reader + +on ECMWF HPC2020 : /hpcperm/c3et/work/python-amip-reader + + +### Download the datasets (deprecated) + +Due to the large size of some of the required files and datasets, they are not provided in this repository. It is necessary to download them from MN4 before executing any code. Use the following script to download the required files (note you must change `bsc32074` to your username in MN4): +```bash +scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/oasis/AMIP/* ./tests/data/forcing +scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/amip-forcing/* ./tests/data/forcing +# PRIMAVERA dataset +scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.2.1/inidata/oasis/AMIP-reader/OLD_AMIP_NL/* ./tests/data/primavera +scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.2.2/inidata/amip-primavera/* ./tests/data/primavera +# Emissions dataset +scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/trunk/inidata/tm5/CMIP6/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412.nc ./tests/data/co2 +``` +If you are downloading the data from PRIMAVERA (+74 GB), it is highly recommended to do it in the background: +```bash +nohup scp -r bsc32074@mn1.bsc.es:/gpfs/projects/bsc32/models/ecearth/v3.2.2/inidata/amip-primavera/* ./tests/data/primavera > nohup.out 2>&1 +# ctrl + z +bg +``` + +Or if running in MN4: +```bash +cp /gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/oasis/AMIP/* ./tests/data/forcing +cp /gpfs/projects/bsc32/models/ecearth/v3.3.3.1/inidata/amip-forcing/* ./tests/data/forcing +cp /gpfs/projects/bsc32/models/ecearth/v3.2.1/inidata/oasis/AMIP-reader/OLD_AMIP_NL/* ./tests/data/primavera +cp /gpfs/projects/bsc32/models/ecearth/v3.2.2/inidata/amip-primavera/* ./tests/data/primavera +cp /gpfs/projects/bsc32/models/ecearth/trunk/inidata/tm5/CMIP6/CO2-em-anthro_input4MIPs_emissions_CMIP_CEDS-2017-05-18_gn_200001-201412.nc ./tests/data/co2 +``` + +### Compile +Each of the example codes is located in its respect folder inside the [`./sources/amip-forcing/`](./sources/amip-forcing/) folder ([`src/fortran/`](./sources/amip-forcing/src/fortran/) and [`src/python/`](./sources/amip-forcing/src/python/)). In the source folder for the Fortran code there is a [Makefile](./sources/amip-forcing/src/fortran/Makefile). Before compiling OASIS, you must create a `make.inc` file which redirects to a make file with your configuration. You may issue the following command to create your own `make.inc` file: +```bash +cp sources/oasis3-mct/util/make_dir/make.templ.inc sources/oasis3-mct/util/make_dir/make.inc +``` + +This would be the contents in `make.inc` if running on your own laptop: +``` +curr_path = $(dir $(realpath $(lastword $(MAKEFILE_LIST)))) +include $(curr_path)/make.linux_gnu_openmpi4 +``` + +For running in BSC Marenostrum4 (mn4), you may change the `include` path to `make.mn4`, so the file contents of `make.inc` would look like this: +``` +curr_path = $(dir $(realpath $(lastword $(MAKEFILE_LIST)))) +include $(curr_path)/make.mn4 +``` + +For running in ECMWF HPC2020 (hpc2020), you may change the `include` path to `make.hpc2020`, so the file contents of `make.inc` would look like this: +``` +curr_path = $(dir $(realpath $(lastword $(MAKEFILE_LIST)))) +include $(curr_path)/make.hpc2020 +``` + +**WARNING: `make.mn4` and `make.hpc2020` assume the proper modules are loaded.** + +For make.mn4 this is done with the following command: `module load python/3.6.1 netcdf/4.2` + +For make.hpc2020 this is done with `module reset ; module load python3/3.10.10-01 prgenv/intel intel/2021.4.0 hpcx-openmpi/2.9.0 hdf5-parallel/1.12.2 netcdf4-parallel/4.9.1` + +For other platforms you may need to modify or add a new makefile for your specific MPI setup. + +By executing the following command, you will compile OASIS, pyOASIS and the Fortran implementation of this model. +```bash +(cd ./sources/amip-forcing/src/fortran && make clean) +(cd ./sources/amip-forcing/src/fortran && make oasis) +(cd ./sources/amip-forcing/src/fortran && make) +``` + +If this is the first time you are compiling pyOASIS in the current session, you must update `LD_LIBRARY_PATH` and `PYTHONPATH` environment variables. Luckily, pyOASIS provides a automatic script: +```bash +source sources/oasis3-mct/generated/python/init.sh +``` +```bash +source sources/oasis3-mct/generated/python/init.csh +``` +Alternatively, these lines could be added to your `~/.bashrc` or `~/.cshrc` file. + +### Run +In order to run the example code with your desired programming language you must execute the script `run_example.sh` inside the [`tests/`](./tests/) folder. Here is an example on how to run the example with the AMIP reader running in Python and the IFS toy model with 4 MPI processes from 1990-01-01 to 1991-01-01 without a fixed year. +```bash +bash ./run_example.sh python forcing 4 1990-01-01 1991-01-01 0 L128 true +``` + +Here is the complete specification for the script: +```bash +./run_example.sh +``` + +Running this script will create a new folder in the root folder of this repository called `work________` with all the output and debug information from the run. + +## Run the tests +Execute the following command in the root folder of this repository to run all tests **except the large test** (make sure to meet all the [requirements](#requirements) stated above). + +If you are on MN4, it is highly recommended to start first an interactive session: +``` +salloc -p interactive +``` + +```bash +bash run_tests.sh +``` +If you also want to run the large test along with the other ones, use the following: +```bash +bash run_tests.sh 1 +``` + +This command will run both implementations (Fortran and Python) of the AMIP reader and check if all of them result in the same output. Here is an example of a successful test run: +``` +Running tests WITHOUT the large test. Use ./run_tests.sh 1 to run all tests including the large test. +***************************************************************** +Running tests... +***************************************************************** +Checking results... +Checking ./work_python_forcing_2_1892-01-01_1894-01-01_0_L128_true vs ./work_fortran_forcing_2_1892-01-01_1894-01-01_0_L128_true +cdo diff: Processed 2 variables over 1462 timesteps [0.23s 52MB]. +cdo diff: Processed 2 variables over 1462 timesteps [0.22s 52MB]. +cdo diff: Processed 2 variables over 1462 timesteps [0.30s 52MB]. +cdo diff: Processed 2 variables over 1462 timesteps [0.31s 52MB]. +... +Tests PASSED +``` + +## Visualize netCDF output files +In order to visualize the results from an experiment, you may first fix the grid and time axis of the `EXPOUT` generated files. First, move into the work directory created when running the experiment. For example: +```bash +cd ./tests/work_python_forcing_4_1990-01-01_1991-01-01_0_L128_true/ +``` + +Then, from inside that directory, run the following script with the start date as the argument: +```bash +bash ./convert-ece4pyreader-grids.sh 19900101 +``` + +As a result, you will now have 8 new files in the working directory for the experiment: `A_Ice_frac.nc`, `A_Ice_frac.grb`, `A_SST.nc`, `A_SST.grb`, `AMIP_sst.nc`, `AMIP_sic.nc`, `areas_AMIP.nc`, `areas_L128.grb`, `areas_L128.nc`, `masks_AMIP.nc`, `masks_L128.grb` and `masks_L128.nc`. + +### Generate GIF files +Optionally, you may want to save visualization as a GIF file. In order to achieve this, you may first run `./convert-ece4pyreader-grids.sh` as before and then the script `nc_to_gif.py` located in the [`tests/`](./tests/) folder. If you are in the working folder and want to visualize the `A_Ice_frac.nc` file, you may run the following command: + +```bash +./nc_to_gif.py work_python_forcing_4_1990-01-01_1991-01-01_0_L128_true A_Ice_frac +``` + +Here is the complete specification for the script: +```bash +./nc_to_gif.py +``` + +Running the script will generate two files: a raw GIF file and an optimized one. diff --git a/rundir/.gitignore b/rundir/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..22e83649f7d5219bb31ce45bd7e986e57ea4e72e --- /dev/null +++ b/rundir/.gitignore @@ -0,0 +1 @@ +/* \ No newline at end of file diff --git a/tests/data/IFS_toy.py b/scripts/IFS_toy.py similarity index 100% rename from tests/data/IFS_toy.py rename to scripts/IFS_toy.py diff --git a/tests/data/convert-ece4pyreader-grids.sh b/scripts/convert-ece4pyreader-grids.sh similarity index 100% rename from tests/data/convert-ece4pyreader-grids.sh rename to scripts/convert-ece4pyreader-grids.sh diff --git a/tests/launch_ece3_data_coupler_hpc2020.cmd b/scripts/launch_ece3_data_coupler_hpc2020.cmd similarity index 100% rename from tests/launch_ece3_data_coupler_hpc2020.cmd rename to scripts/launch_ece3_data_coupler_hpc2020.cmd diff --git a/tests/launch_ece3_data_coupler_mn4.cmd b/scripts/launch_ece3_data_coupler_mn4.cmd similarity index 100% rename from tests/launch_ece3_data_coupler_mn4.cmd rename to scripts/launch_ece3_data_coupler_mn4.cmd diff --git a/tests/launch_ece3_toy_model_hpc2020.cmd b/scripts/launch_ece3_toy_model_hpc2020.cmd similarity index 100% rename from tests/launch_ece3_toy_model_hpc2020.cmd rename to scripts/launch_ece3_toy_model_hpc2020.cmd diff --git a/tests/launch_ece3_toy_model_mn4.cmd b/scripts/launch_ece3_toy_model_mn4.cmd similarity index 100% rename from tests/launch_ece3_toy_model_mn4.cmd rename to scripts/launch_ece3_toy_model_mn4.cmd diff --git a/tests/modules.sh b/scripts/modules.sh similarity index 100% rename from tests/modules.sh rename to scripts/modules.sh diff --git a/tests/data/namcouple.sh b/scripts/namcouple.sh similarity index 100% rename from tests/data/namcouple.sh rename to scripts/namcouple.sh diff --git a/tests/data/namcouple_co2.sh b/scripts/namcouple_co2.sh similarity index 100% rename from tests/data/namcouple_co2.sh rename to scripts/namcouple_co2.sh diff --git a/tests/data/namcouple_co2box.sh b/scripts/namcouple_co2box.sh similarity index 100% rename from tests/data/namcouple_co2box.sh rename to scripts/namcouple_co2box.sh diff --git a/tests/data/namcouple_primavera.sh b/scripts/namcouple_primavera.sh similarity index 100% rename from tests/data/namcouple_primavera.sh rename to scripts/namcouple_primavera.sh diff --git a/tests/data/namelist.amip.sh b/scripts/namelist.amip.sh similarity index 100% rename from tests/data/namelist.amip.sh rename to scripts/namelist.amip.sh diff --git a/tests/data/namelist.co2box.sh b/scripts/namelist.co2box.sh similarity index 100% rename from tests/data/namelist.co2box.sh rename to scripts/namelist.co2box.sh diff --git a/tests/data/namelist_primavera.amip.sh b/scripts/namelist_primavera.amip.sh similarity index 100% rename from tests/data/namelist_primavera.amip.sh rename to scripts/namelist_primavera.amip.sh diff --git a/tests/nc_to_gif.py b/scripts/nc_to_gif.py similarity index 100% rename from tests/nc_to_gif.py rename to scripts/nc_to_gif.py diff --git a/tests/run_example.sh b/scripts/run_example.sh similarity index 95% rename from tests/run_example.sh rename to scripts/run_example.sh index f88c4d187998940c3a779fd30151816be84bfa77..9afa10e1ba9f197ea8a89d7dc3e99831f1b06840 100755 --- a/tests/run_example.sh +++ b/scripts/run_example.sh @@ -28,7 +28,7 @@ user=`whoami` ## - Define paths srcdir=`pwd` -datadir=$srcdir/data +datadir=$srcdir/../data # ece_3_toy_model as exe1 exe1='python3 toy_model.py' @@ -57,7 +57,7 @@ else fi # - Define rundir -rundir=${srcdir}/work_${model}_${model_mode}_${nproc_exe1}_${leg_start_date}_${leg_end_date}_${ifs_cmip_fixyear}_${ifs_grid}_${amip_interpolate} +rundir=${srcdir}/../rundir/work_${model}_${model_mode}_${nproc_exe1}_${leg_start_date}_${leg_end_date}_${ifs_cmip_fixyear}_${ifs_grid}_${amip_interpolate} echo '*****************************************************************' @@ -108,12 +108,13 @@ else cp -f $datadir/masks-noamip.nc $rundir/masks.nc cp -f $datadir/areas-noamip.nc $rundir/areas.nc fi -#cp -f $datadir/*.sh $rundir/. -cp -f $datadir/convert-ece4pyreader-grids.sh $rundir/. -cp -f $datadir/*.yaml $rundir/. + +cp -f $srcdir/convert-ece4pyreader-grids.sh $rundir/. ln -sf $source_exe1 $rundir/. ln -sf $source_exe2 $rundir/. cp -f $srcdir/*.py $rundir/. +cp -f $srcdir/*.yaml $rundir/. +cp -f $srcdir/*.sh $rundir/. cd $rundir ###################################################################### diff --git a/run_tests.sh b/scripts/run_tests.sh similarity index 100% rename from run_tests.sh rename to scripts/run_tests.sh diff --git a/tests/run_toy_model.sh b/scripts/run_toy_model.sh similarity index 72% rename from tests/run_toy_model.sh rename to scripts/run_toy_model.sh index cc215814e2fea382df8c13cb913e9d6ec168ceaf..de76ba2b4590f24b399511bae96c7f88d915377c 100755 --- a/tests/run_toy_model.sh +++ b/scripts/run_toy_model.sh @@ -8,11 +8,11 @@ set -xuve exp_id="ifs_nemo_runoff" # prepare rundir -testdir=`pwd` -datadir=$testdir/data/ece3-toy-model -srcdir=$testdir/../sources/ece3-toy-model +srcdir=`pwd` +datadir=$srcdir/../data/ece3-toy-model +toydir=$srcdir/../sources/ece3-toy-model -rundir=${testdir}/work_ece3_toy_model_${exp_id} +rundir=${srcdir}/../rundir/work_ece3_toy_model_${exp_id} rm -rf $rundir mkdir -p $rundir @@ -21,9 +21,9 @@ for f in masks.nc grids.nc areas.nc rstas.nc rstos.nc; do cp -f $datadir/$f $rundir done cp -f $datadir/rmp* $rundir -cp -f $srcdir/*.py $rundir -cp -f $srcdir/namcouple $rundir -cp -f $srcdir/*.yaml $rundir +cp -f $toydir/*.py $rundir +cp -f $toydir/namcouple $rundir +cp -f $toydir/*.yaml $rundir toy_model="toy_model.py" diff --git a/tests/data/template_conf_datacoupler_co2box.yaml b/scripts/template_conf_datacoupler_co2box.yaml similarity index 100% rename from tests/data/template_conf_datacoupler_co2box.yaml rename to scripts/template_conf_datacoupler_co2box.yaml diff --git a/tests/data/template_conf_datacoupler_co2flux.yaml b/scripts/template_conf_datacoupler_co2flux.yaml similarity index 100% rename from tests/data/template_conf_datacoupler_co2flux.yaml rename to scripts/template_conf_datacoupler_co2flux.yaml diff --git a/tests/data/template_conf_datacoupler_forcing.yaml b/scripts/template_conf_datacoupler_forcing.yaml similarity index 100% rename from tests/data/template_conf_datacoupler_forcing.yaml rename to scripts/template_conf_datacoupler_forcing.yaml diff --git a/sources/data-coupler/how_to_run_mn4.ipynb b/sources/data-coupler/how_to_run_mn4.ipynb deleted file mode 100644 index 48b592b992123719e7c42dc609966458de2150ea..0000000000000000000000000000000000000000 --- a/sources/data-coupler/how_to_run_mn4.ipynb +++ /dev/null @@ -1,171 +0,0 @@ -{ - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "# Introduction \n", - "The Data Coupler is being developed at the BSC to implement a CO2 box model for EC-Earth3 and to read and interpolate CO2 anthropogenic emissions (from e.g. CMIP6 input4MIPS) and CO2 surface fluxes (from e.g. vegetation+fire+ocean fluxes from SiB4). \n", - "\n", - "## Getting the model \n", - "The model can be cloned from (BSC-ES gitlab repository)[https://earth.bsc.es/gitlab/es/python-amip-reader] \n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "git clone https://earth.bsc.es/gitlab/es/python-amip-reader.git\n", - "cd python-amip-reader\n", - "git checkout data-coupler" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Setting up the model\n", - "First copy the datasets from Etienne's directory ```/gpfs/scratch/bsc32/bsc32051/git/python-amip-reader/tests/data/```" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "datadir=/gpfs/scratch/bsc32/bsc32051/git/python-amip-reader/tests/data/\n", - "cd /python-amip-reader\n", - "cp -r $datadir/forcing/* tests/data/forcing\n", - "cp -r $datadir/co2/* tests/data/co2" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To build oasis and the optional fortran amip-forcing follow the instructions in https://earth.bsc.es/gitlab/es/python-amip-reader/-/blob/data-coupler/README.md#compile" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cd /sources/oasis3-mct/util/make_dir\n", - "cp make.templ.inc make.inc" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Please chnage the path of the make file to include MN4, and then complie the as follows: " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "include $(curr_path)/make.mn4\n", - "\n", - " \n", - "cd ../sources/amip-forcing/src/fortran\n", - " \n", - "module load impi/2017.4\n", - "module load mkl/2017.4\n", - "module load intel/2021.4\n", - "module load python/3.6.1\n", - "module load netcdf/4.2\n", - "module load cdo/1.9.3\n", - "\n", - "make realclean && make oasis && make" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Running the model\n", - "To run the model you need to edit file $HPCPERM/work/python-amip-reader/tests/launch-hpc2020.cmd and make sure the proper configuration is un-commented at the end, for example\n", - "\n", - "For a simple AMIP run using the classic fortran amip-forcing: \n", - "\n", - "bash ./run_example.sh fortran forcing 4 2005-01-01 2005-03-01 0 128 true false\n", - "\n", - "For the CO2 box model :\n", - "\n", - "bash ./run_example.sh co2box co2 4 2005-01-01 2005-05-01 0 128 true false\n", - "\n", - "The following lines hard-coded in run_example.sh control the options of the CO2 box model\n", - "\n", - " CO2_CMODE=true # set to false to disable co2 concentrations reading\n", - " CO2_EMODE=true # set to false to disable emissions reading\n", - "\n", - "\n", - "\n", - "There are 2 ways to run the model: \n", - "\n", - "1) run an interactive session and launch the job with bash (for short runs and debugging)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "ecinteractive -c6 -t 12:00:00 -m 32GB\n", - "cd $HPCPERM/work/python-amip-reader/tests/\n", - "bash launch-hpc2020.cmd" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "2) launch the job with sbatch (for longer runs)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cd $HPCPERM/work/python-amip-reader/tests/\n", - "sbatch launch-hpc2020.cmd" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If the run is successful, output will be present in a folder representing the run you made e.g. work_fortran_forcing_4_2005-01-01_2005-03-01_0_128_true\n", - "\n", - "Some files are post-processed with proper grid and time metadata applied." - ] - } - ], - "metadata": { - "language_info": { - "name": "python" - }, - "orig_nbformat": 4 - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/tests/data/co2/.gitignore b/tests/data/co2/.gitignore deleted file mode 100644 index c96a04f008ee21e260b28f7701595ed59e2839e3..0000000000000000000000000000000000000000 --- a/tests/data/co2/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -* -!.gitignore \ No newline at end of file diff --git a/tests/data/ece3-toy-model/grid-runoff.txt b/tests/data/ece3-toy-model/grid-runoff.txt deleted file mode 100644 index 5cdc9aca11e6a12a9b8faaa85703d2f1877ef573..0000000000000000000000000000000000000000 --- a/tests/data/ece3-toy-model/grid-runoff.txt +++ /dev/null @@ -1,7 +0,0 @@ -# -# gridID 1 -# -gridtype = generic -gridsize = 131072 -xsize = 512 -ysize = 256 diff --git a/tests/data/forcing/.gitignore b/tests/data/forcing/.gitignore deleted file mode 100644 index c96a04f008ee21e260b28f7701595ed59e2839e3..0000000000000000000000000000000000000000 --- a/tests/data/forcing/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -* -!.gitignore \ No newline at end of file diff --git a/tests/data/grid-amip.txt b/tests/data/grid-amip.txt deleted file mode 100644 index 4b4aa9c4d40557652544711e49aa82e225318936..0000000000000000000000000000000000000000 --- a/tests/data/grid-amip.txt +++ /dev/null @@ -1,558 +0,0 @@ -# -# gridID 1 -# -gridtype = lonlat -gridsize = 64800 -xsize = 360 -ysize = 180 -xname = lon -xlongname = "longitude" -xunits = "degrees_east" -yname = lat -ylongname = "latitude" -yunits = "degrees_north" -xfirst = 0.5 -xinc = 1 -xbounds = 0 1 - 1 2 - 2 3 - 3 4 - 4 5 - 5 6 - 6 7 - 7 8 - 8 9 - 9 10 - 10 11 - 11 12 - 12 13 - 13 14 - 14 15 - 15 16 - 16 17 - 17 18 - 18 19 - 19 20 - 20 21 - 21 22 - 22 23 - 23 24 - 24 25 - 25 26 - 26 27 - 27 28 - 28 29 - 29 30 - 30 31 - 31 32 - 32 33 - 33 34 - 34 35 - 35 36 - 36 37 - 37 38 - 38 39 - 39 40 - 40 41 - 41 42 - 42 43 - 43 44 - 44 45 - 45 46 - 46 47 - 47 48 - 48 49 - 49 50 - 50 51 - 51 52 - 52 53 - 53 54 - 54 55 - 55 56 - 56 57 - 57 58 - 58 59 - 59 60 - 60 61 - 61 62 - 62 63 - 63 64 - 64 65 - 65 66 - 66 67 - 67 68 - 68 69 - 69 70 - 70 71 - 71 72 - 72 73 - 73 74 - 74 75 - 75 76 - 76 77 - 77 78 - 78 79 - 79 80 - 80 81 - 81 82 - 82 83 - 83 84 - 84 85 - 85 86 - 86 87 - 87 88 - 88 89 - 89 90 - 90 91 - 91 92 - 92 93 - 93 94 - 94 95 - 95 96 - 96 97 - 97 98 - 98 99 - 99 100 - 100 101 - 101 102 - 102 103 - 103 104 - 104 105 - 105 106 - 106 107 - 107 108 - 108 109 - 109 110 - 110 111 - 111 112 - 112 113 - 113 114 - 114 115 - 115 116 - 116 117 - 117 118 - 118 119 - 119 120 - 120 121 - 121 122 - 122 123 - 123 124 - 124 125 - 125 126 - 126 127 - 127 128 - 128 129 - 129 130 - 130 131 - 131 132 - 132 133 - 133 134 - 134 135 - 135 136 - 136 137 - 137 138 - 138 139 - 139 140 - 140 141 - 141 142 - 142 143 - 143 144 - 144 145 - 145 146 - 146 147 - 147 148 - 148 149 - 149 150 - 150 151 - 151 152 - 152 153 - 153 154 - 154 155 - 155 156 - 156 157 - 157 158 - 158 159 - 159 160 - 160 161 - 161 162 - 162 163 - 163 164 - 164 165 - 165 166 - 166 167 - 167 168 - 168 169 - 169 170 - 170 171 - 171 172 - 172 173 - 173 174 - 174 175 - 175 176 - 176 177 - 177 178 - 178 179 - 179 180 - 180 181 - 181 182 - 182 183 - 183 184 - 184 185 - 185 186 - 186 187 - 187 188 - 188 189 - 189 190 - 190 191 - 191 192 - 192 193 - 193 194 - 194 195 - 195 196 - 196 197 - 197 198 - 198 199 - 199 200 - 200 201 - 201 202 - 202 203 - 203 204 - 204 205 - 205 206 - 206 207 - 207 208 - 208 209 - 209 210 - 210 211 - 211 212 - 212 213 - 213 214 - 214 215 - 215 216 - 216 217 - 217 218 - 218 219 - 219 220 - 220 221 - 221 222 - 222 223 - 223 224 - 224 225 - 225 226 - 226 227 - 227 228 - 228 229 - 229 230 - 230 231 - 231 232 - 232 233 - 233 234 - 234 235 - 235 236 - 236 237 - 237 238 - 238 239 - 239 240 - 240 241 - 241 242 - 242 243 - 243 244 - 244 245 - 245 246 - 246 247 - 247 248 - 248 249 - 249 250 - 250 251 - 251 252 - 252 253 - 253 254 - 254 255 - 255 256 - 256 257 - 257 258 - 258 259 - 259 260 - 260 261 - 261 262 - 262 263 - 263 264 - 264 265 - 265 266 - 266 267 - 267 268 - 268 269 - 269 270 - 270 271 - 271 272 - 272 273 - 273 274 - 274 275 - 275 276 - 276 277 - 277 278 - 278 279 - 279 280 - 280 281 - 281 282 - 282 283 - 283 284 - 284 285 - 285 286 - 286 287 - 287 288 - 288 289 - 289 290 - 290 291 - 291 292 - 292 293 - 293 294 - 294 295 - 295 296 - 296 297 - 297 298 - 298 299 - 299 300 - 300 301 - 301 302 - 302 303 - 303 304 - 304 305 - 305 306 - 306 307 - 307 308 - 308 309 - 309 310 - 310 311 - 311 312 - 312 313 - 313 314 - 314 315 - 315 316 - 316 317 - 317 318 - 318 319 - 319 320 - 320 321 - 321 322 - 322 323 - 323 324 - 324 325 - 325 326 - 326 327 - 327 328 - 328 329 - 329 330 - 330 331 - 331 332 - 332 333 - 333 334 - 334 335 - 335 336 - 336 337 - 337 338 - 338 339 - 339 340 - 340 341 - 341 342 - 342 343 - 343 344 - 344 345 - 345 346 - 346 347 - 347 348 - 348 349 - 349 350 - 350 351 - 351 352 - 352 353 - 353 354 - 354 355 - 355 356 - 356 357 - 357 358 - 358 359 - 359 360 -yfirst = -89.5 -yinc = 1 -ybounds = -90 -89 - -89 -88 - -88 -87 - -87 -86 - -86 -85 - -85 -84 - -84 -83 - -83 -82 - -82 -81 - -81 -80 - -80 -79 - -79 -78 - -78 -77 - -77 -76 - -76 -75 - -75 -74 - -74 -73 - -73 -72 - -72 -71 - -71 -70 - -70 -69 - -69 -68 - -68 -67 - -67 -66 - -66 -65 - -65 -64 - -64 -63 - -63 -62 - -62 -61 - -61 -60 - -60 -59 - -59 -58 - -58 -57 - -57 -56 - -56 -55 - -55 -54 - -54 -53 - -53 -52 - -52 -51 - -51 -50 - -50 -49 - -49 -48 - -48 -47 - -47 -46 - -46 -45 - -45 -44 - -44 -43 - -43 -42 - -42 -41 - -41 -40 - -40 -39 - -39 -38 - -38 -37 - -37 -36 - -36 -35 - -35 -34 - -34 -33 - -33 -32 - -32 -31 - -31 -30 - -30 -29 - -29 -28 - -28 -27 - -27 -26 - -26 -25 - -25 -24 - -24 -23 - -23 -22 - -22 -21 - -21 -20 - -20 -19 - -19 -18 - -18 -17 - -17 -16 - -16 -15 - -15 -14 - -14 -13 - -13 -12 - -12 -11 - -11 -10 - -10 -9 - -9 -8 - -8 -7 - -7 -6 - -6 -5 - -5 -4 - -4 -3 - -3 -2 - -2 -1 - -1 0 - 0 1 - 1 2 - 2 3 - 3 4 - 4 5 - 5 6 - 6 7 - 7 8 - 8 9 - 9 10 - 10 11 - 11 12 - 12 13 - 13 14 - 14 15 - 15 16 - 16 17 - 17 18 - 18 19 - 19 20 - 20 21 - 21 22 - 22 23 - 23 24 - 24 25 - 25 26 - 26 27 - 27 28 - 28 29 - 29 30 - 30 31 - 31 32 - 32 33 - 33 34 - 34 35 - 35 36 - 36 37 - 37 38 - 38 39 - 39 40 - 40 41 - 41 42 - 42 43 - 43 44 - 44 45 - 45 46 - 46 47 - 47 48 - 48 49 - 49 50 - 50 51 - 51 52 - 52 53 - 53 54 - 54 55 - 55 56 - 56 57 - 57 58 - 58 59 - 59 60 - 60 61 - 61 62 - 62 63 - 63 64 - 64 65 - 65 66 - 66 67 - 67 68 - 68 69 - 69 70 - 70 71 - 71 72 - 72 73 - 73 74 - 74 75 - 75 76 - 76 77 - 77 78 - 78 79 - 79 80 - 80 81 - 81 82 - 82 83 - 83 84 - 84 85 - 85 86 - 86 87 - 87 88 - 88 89 - 89 90 -scanningMode = 64 diff --git a/tests/data/grid-co2_emis.txt b/tests/data/grid-co2_emis.txt deleted file mode 100644 index a392e691ee2d7c541c58de2afea130fb4367e638..0000000000000000000000000000000000000000 --- a/tests/data/grid-co2_emis.txt +++ /dev/null @@ -1,1097 +0,0 @@ -# -# gridID 1 -# -gridtype = lonlat -gridsize = 259200 -xsize = 720 -ysize = 360 -xname = lon -xlongname = "longitude" -xunits = "degrees_east" -yname = lat -ylongname = "latitude" -yunits = "degrees_north" -xfirst = -179.75 -xinc = 0.5 -xbounds = -180 -179.5 - -179.5 -179 - -179 -178.5 - -178.5 -178 - -178 -177.5 - -177.5 -177 - -177 -176.5 - -176.5 -176 - -176 -175.5 - -175.5 -175 - -175 -174.5 - -174.5 -174 - -174 -173.5 - -173.5 -173 - -173 -172.5 - -172.5 -172 - -172 -171.5 - -171.5 -171 - -171 -170.5 - -170.5 -170 - -170 -169.5 - -169.5 -169 - -169 -168.5 - -168.5 -168 - -168 -167.5 - -167.5 -167 - -167 -166.5 - -166.5 -166 - -166 -165.5 - -165.5 -165 - -165 -164.5 - -164.5 -164 - -164 -163.5 - -163.5 -163 - -163 -162.5 - -162.5 -162 - -162 -161.5 - -161.5 -161 - -161 -160.5 - -160.5 -160 - -160 -159.5 - -159.5 -159 - -159 -158.5 - -158.5 -158 - -158 -157.5 - -157.5 -157 - -157 -156.5 - -156.5 -156 - -156 -155.5 - -155.5 -155 - -155 -154.5 - -154.5 -154 - -154 -153.5 - -153.5 -153 - -153 -152.5 - -152.5 -152 - -152 -151.5 - -151.5 -151 - -151 -150.5 - -150.5 -150 - -150 -149.5 - -149.5 -149 - -149 -148.5 - -148.5 -148 - -148 -147.5 - -147.5 -147 - -147 -146.5 - -146.5 -146 - -146 -145.5 - -145.5 -145 - -145 -144.5 - -144.5 -144 - -144 -143.5 - -143.5 -143 - -143 -142.5 - -142.5 -142 - -142 -141.5 - -141.5 -141 - -141 -140.5 - -140.5 -140 - -140 -139.5 - -139.5 -139 - -139 -138.5 - -138.5 -138 - -138 -137.5 - -137.5 -137 - -137 -136.5 - -136.5 -136 - -136 -135.5 - -135.5 -135 - -135 -134.5 - -134.5 -134 - -134 -133.5 - -133.5 -133 - -133 -132.5 - -132.5 -132 - -132 -131.5 - -131.5 -131 - -131 -130.5 - -130.5 -130 - -130 -129.5 - -129.5 -129 - -129 -128.5 - -128.5 -128 - -128 -127.5 - -127.5 -127 - -127 -126.5 - -126.5 -126 - -126 -125.5 - -125.5 -125 - -125 -124.5 - -124.5 -124 - -124 -123.5 - -123.5 -123 - -123 -122.5 - -122.5 -122 - -122 -121.5 - -121.5 -121 - -121 -120.5 - -120.5 -120 - -120 -119.5 - -119.5 -119 - -119 -118.5 - -118.5 -118 - -118 -117.5 - -117.5 -117 - -117 -116.5 - -116.5 -116 - -116 -115.5 - -115.5 -115 - -115 -114.5 - -114.5 -114 - -114 -113.5 - -113.5 -113 - -113 -112.5 - -112.5 -112 - -112 -111.5 - -111.5 -111 - -111 -110.5 - -110.5 -110 - -110 -109.5 - -109.5 -109 - -109 -108.5 - -108.5 -108 - -108 -107.5 - -107.5 -107 - -107 -106.5 - -106.5 -106 - -106 -105.5 - -105.5 -105 - -105 -104.5 - -104.5 -104 - -104 -103.5 - -103.5 -103 - -103 -102.5 - -102.5 -102 - -102 -101.5 - -101.5 -101 - -101 -100.5 - -100.5 -100 - -100 -99.5 - -99.5 -99 - -99 -98.5 - -98.5 -98 - -98 -97.5 - -97.5 -97 - -97 -96.5 - -96.5 -96 - -96 -95.5 - -95.5 -95 - -95 -94.5 - -94.5 -94 - -94 -93.5 - -93.5 -93 - -93 -92.5 - -92.5 -92 - -92 -91.5 - -91.5 -91 - -91 -90.5 - -90.5 -90 - -90 -89.5 - -89.5 -89 - -89 -88.5 - -88.5 -88 - -88 -87.5 - -87.5 -87 - -87 -86.5 - -86.5 -86 - -86 -85.5 - -85.5 -85 - -85 -84.5 - -84.5 -84 - -84 -83.5 - -83.5 -83 - -83 -82.5 - -82.5 -82 - -82 -81.5 - -81.5 -81 - -81 -80.5 - -80.5 -80 - -80 -79.5 - -79.5 -79 - -79 -78.5 - -78.5 -78 - -78 -77.5 - -77.5 -77 - -77 -76.5 - -76.5 -76 - -76 -75.5 - -75.5 -75 - -75 -74.5 - -74.5 -74 - -74 -73.5 - -73.5 -73 - -73 -72.5 - -72.5 -72 - -72 -71.5 - -71.5 -71 - -71 -70.5 - -70.5 -70 - -70 -69.5 - -69.5 -69 - -69 -68.5 - -68.5 -68 - -68 -67.5 - -67.5 -67 - -67 -66.5 - -66.5 -66 - -66 -65.5 - -65.5 -65 - -65 -64.5 - -64.5 -64 - -64 -63.5 - -63.5 -63 - -63 -62.5 - -62.5 -62 - -62 -61.5 - -61.5 -61 - -61 -60.5 - -60.5 -60 - -60 -59.5 - -59.5 -59 - -59 -58.5 - -58.5 -58 - -58 -57.5 - -57.5 -57 - -57 -56.5 - -56.5 -56 - -56 -55.5 - -55.5 -55 - -55 -54.5 - -54.5 -54 - -54 -53.5 - -53.5 -53 - -53 -52.5 - -52.5 -52 - -52 -51.5 - -51.5 -51 - -51 -50.5 - -50.5 -50 - -50 -49.5 - -49.5 -49 - -49 -48.5 - -48.5 -48 - -48 -47.5 - -47.5 -47 - -47 -46.5 - -46.5 -46 - -46 -45.5 - -45.5 -45 - -45 -44.5 - -44.5 -44 - -44 -43.5 - -43.5 -43 - -43 -42.5 - -42.5 -42 - -42 -41.5 - -41.5 -41 - -41 -40.5 - -40.5 -40 - -40 -39.5 - -39.5 -39 - -39 -38.5 - -38.5 -38 - -38 -37.5 - -37.5 -37 - -37 -36.5 - -36.5 -36 - -36 -35.5 - -35.5 -35 - -35 -34.5 - -34.5 -34 - -34 -33.5 - -33.5 -33 - -33 -32.5 - -32.5 -32 - -32 -31.5 - -31.5 -31 - -31 -30.5 - -30.5 -30 - -30 -29.5 - -29.5 -29 - -29 -28.5 - -28.5 -28 - -28 -27.5 - -27.5 -27 - -27 -26.5 - -26.5 -26 - -26 -25.5 - -25.5 -25 - -25 -24.5 - -24.5 -24 - -24 -23.5 - -23.5 -23 - -23 -22.5 - -22.5 -22 - -22 -21.5 - -21.5 -21 - -21 -20.5 - -20.5 -20 - -20 -19.5 - -19.5 -19 - -19 -18.5 - -18.5 -18 - -18 -17.5 - -17.5 -17 - -17 -16.5 - -16.5 -16 - -16 -15.5 - -15.5 -15 - -15 -14.5 - -14.5 -14 - -14 -13.5 - -13.5 -13 - -13 -12.5 - -12.5 -12 - -12 -11.5 - -11.5 -11 - -11 -10.5 - -10.5 -10 - -10 -9.5 - -9.5 -9 - -9 -8.5 - -8.5 -8 - -8 -7.5 - -7.5 -7 - -7 -6.5 - -6.5 -6 - -6 -5.5 - -5.5 -5 - -5 -4.5 - -4.5 -4 - -4 -3.5 - -3.5 -3 - -3 -2.5 - -2.5 -2 - -2 -1.5 - -1.5 -1 - -1 -0.5 - -0.5 0 - 0 0.5 - 0.5 1 - 1 1.5 - 1.5 2 - 2 2.5 - 2.5 3 - 3 3.5 - 3.5 4 - 4 4.5 - 4.5 5 - 5 5.5 - 5.5 6 - 6 6.5 - 6.5 7 - 7 7.5 - 7.5 8 - 8 8.5 - 8.5 9 - 9 9.5 - 9.5 10 - 10 10.5 - 10.5 11 - 11 11.5 - 11.5 12 - 12 12.5 - 12.5 13 - 13 13.5 - 13.5 14 - 14 14.5 - 14.5 15 - 15 15.5 - 15.5 16 - 16 16.5 - 16.5 17 - 17 17.5 - 17.5 18 - 18 18.5 - 18.5 19 - 19 19.5 - 19.5 20 - 20 20.5 - 20.5 21 - 21 21.5 - 21.5 22 - 22 22.5 - 22.5 23 - 23 23.5 - 23.5 24 - 24 24.5 - 24.5 25 - 25 25.5 - 25.5 26 - 26 26.5 - 26.5 27 - 27 27.5 - 27.5 28 - 28 28.5 - 28.5 29 - 29 29.5 - 29.5 30 - 30 30.5 - 30.5 31 - 31 31.5 - 31.5 32 - 32 32.5 - 32.5 33 - 33 33.5 - 33.5 34 - 34 34.5 - 34.5 35 - 35 35.5 - 35.5 36 - 36 36.5 - 36.5 37 - 37 37.5 - 37.5 38 - 38 38.5 - 38.5 39 - 39 39.5 - 39.5 40 - 40 40.5 - 40.5 41 - 41 41.5 - 41.5 42 - 42 42.5 - 42.5 43 - 43 43.5 - 43.5 44 - 44 44.5 - 44.5 45 - 45 45.5 - 45.5 46 - 46 46.5 - 46.5 47 - 47 47.5 - 47.5 48 - 48 48.5 - 48.5 49 - 49 49.5 - 49.5 50 - 50 50.5 - 50.5 51 - 51 51.5 - 51.5 52 - 52 52.5 - 52.5 53 - 53 53.5 - 53.5 54 - 54 54.5 - 54.5 55 - 55 55.5 - 55.5 56 - 56 56.5 - 56.5 57 - 57 57.5 - 57.5 58 - 58 58.5 - 58.5 59 - 59 59.5 - 59.5 60 - 60 60.5 - 60.5 61 - 61 61.5 - 61.5 62 - 62 62.5 - 62.5 63 - 63 63.5 - 63.5 64 - 64 64.5 - 64.5 65 - 65 65.5 - 65.5 66 - 66 66.5 - 66.5 67 - 67 67.5 - 67.5 68 - 68 68.5 - 68.5 69 - 69 69.5 - 69.5 70 - 70 70.5 - 70.5 71 - 71 71.5 - 71.5 72 - 72 72.5 - 72.5 73 - 73 73.5 - 73.5 74 - 74 74.5 - 74.5 75 - 75 75.5 - 75.5 76 - 76 76.5 - 76.5 77 - 77 77.5 - 77.5 78 - 78 78.5 - 78.5 79 - 79 79.5 - 79.5 80 - 80 80.5 - 80.5 81 - 81 81.5 - 81.5 82 - 82 82.5 - 82.5 83 - 83 83.5 - 83.5 84 - 84 84.5 - 84.5 85 - 85 85.5 - 85.5 86 - 86 86.5 - 86.5 87 - 87 87.5 - 87.5 88 - 88 88.5 - 88.5 89 - 89 89.5 - 89.5 90 - 90 90.5 - 90.5 91 - 91 91.5 - 91.5 92 - 92 92.5 - 92.5 93 - 93 93.5 - 93.5 94 - 94 94.5 - 94.5 95 - 95 95.5 - 95.5 96 - 96 96.5 - 96.5 97 - 97 97.5 - 97.5 98 - 98 98.5 - 98.5 99 - 99 99.5 - 99.5 100 - 100 100.5 - 100.5 101 - 101 101.5 - 101.5 102 - 102 102.5 - 102.5 103 - 103 103.5 - 103.5 104 - 104 104.5 - 104.5 105 - 105 105.5 - 105.5 106 - 106 106.5 - 106.5 107 - 107 107.5 - 107.5 108 - 108 108.5 - 108.5 109 - 109 109.5 - 109.5 110 - 110 110.5 - 110.5 111 - 111 111.5 - 111.5 112 - 112 112.5 - 112.5 113 - 113 113.5 - 113.5 114 - 114 114.5 - 114.5 115 - 115 115.5 - 115.5 116 - 116 116.5 - 116.5 117 - 117 117.5 - 117.5 118 - 118 118.5 - 118.5 119 - 119 119.5 - 119.5 120 - 120 120.5 - 120.5 121 - 121 121.5 - 121.5 122 - 122 122.5 - 122.5 123 - 123 123.5 - 123.5 124 - 124 124.5 - 124.5 125 - 125 125.5 - 125.5 126 - 126 126.5 - 126.5 127 - 127 127.5 - 127.5 128 - 128 128.5 - 128.5 129 - 129 129.5 - 129.5 130 - 130 130.5 - 130.5 131 - 131 131.5 - 131.5 132 - 132 132.5 - 132.5 133 - 133 133.5 - 133.5 134 - 134 134.5 - 134.5 135 - 135 135.5 - 135.5 136 - 136 136.5 - 136.5 137 - 137 137.5 - 137.5 138 - 138 138.5 - 138.5 139 - 139 139.5 - 139.5 140 - 140 140.5 - 140.5 141 - 141 141.5 - 141.5 142 - 142 142.5 - 142.5 143 - 143 143.5 - 143.5 144 - 144 144.5 - 144.5 145 - 145 145.5 - 145.5 146 - 146 146.5 - 146.5 147 - 147 147.5 - 147.5 148 - 148 148.5 - 148.5 149 - 149 149.5 - 149.5 150 - 150 150.5 - 150.5 151 - 151 151.5 - 151.5 152 - 152 152.5 - 152.5 153 - 153 153.5 - 153.5 154 - 154 154.5 - 154.5 155 - 155 155.5 - 155.5 156 - 156 156.5 - 156.5 157 - 157 157.5 - 157.5 158 - 158 158.5 - 158.5 159 - 159 159.5 - 159.5 160 - 160 160.5 - 160.5 161 - 161 161.5 - 161.5 162 - 162 162.5 - 162.5 163 - 163 163.5 - 163.5 164 - 164 164.5 - 164.5 165 - 165 165.5 - 165.5 166 - 166 166.5 - 166.5 167 - 167 167.5 - 167.5 168 - 168 168.5 - 168.5 169 - 169 169.5 - 169.5 170 - 170 170.5 - 170.5 171 - 171 171.5 - 171.5 172 - 172 172.5 - 172.5 173 - 173 173.5 - 173.5 174 - 174 174.5 - 174.5 175 - 175 175.5 - 175.5 176 - 176 176.5 - 176.5 177 - 177 177.5 - 177.5 178 - 178 178.5 - 178.5 179 - 179 179.5 - 179.5 180 -yfirst = -89.75 -yinc = 0.5 -ybounds = -90 -89.5 - -89.5 -89 - -89 -88.5 - -88.5 -88 - -88 -87.5 - -87.5 -87 - -87 -86.5 - -86.5 -86 - -86 -85.5 - -85.5 -85 - -85 -84.5 - -84.5 -84 - -84 -83.5 - -83.5 -83 - -83 -82.5 - -82.5 -82 - -82 -81.5 - -81.5 -81 - -81 -80.5 - -80.5 -80 - -80 -79.5 - -79.5 -79 - -79 -78.5 - -78.5 -78 - -78 -77.5 - -77.5 -77 - -77 -76.5 - -76.5 -76 - -76 -75.5 - -75.5 -75 - -75 -74.5 - -74.5 -74 - -74 -73.5 - -73.5 -73 - -73 -72.5 - -72.5 -72 - -72 -71.5 - -71.5 -71 - -71 -70.5 - -70.5 -70 - -70 -69.5 - -69.5 -69 - -69 -68.5 - -68.5 -68 - -68 -67.5 - -67.5 -67 - -67 -66.5 - -66.5 -66 - -66 -65.5 - -65.5 -65 - -65 -64.5 - -64.5 -64 - -64 -63.5 - -63.5 -63 - -63 -62.5 - -62.5 -62 - -62 -61.5 - -61.5 -61 - -61 -60.5 - -60.5 -60 - -60 -59.5 - -59.5 -59 - -59 -58.5 - -58.5 -58 - -58 -57.5 - -57.5 -57 - -57 -56.5 - -56.5 -56 - -56 -55.5 - -55.5 -55 - -55 -54.5 - -54.5 -54 - -54 -53.5 - -53.5 -53 - -53 -52.5 - -52.5 -52 - -52 -51.5 - -51.5 -51 - -51 -50.5 - -50.5 -50 - -50 -49.5 - -49.5 -49 - -49 -48.5 - -48.5 -48 - -48 -47.5 - -47.5 -47 - -47 -46.5 - -46.5 -46 - -46 -45.5 - -45.5 -45 - -45 -44.5 - -44.5 -44 - -44 -43.5 - -43.5 -43 - -43 -42.5 - -42.5 -42 - -42 -41.5 - -41.5 -41 - -41 -40.5 - -40.5 -40 - -40 -39.5 - -39.5 -39 - -39 -38.5 - -38.5 -38 - -38 -37.5 - -37.5 -37 - -37 -36.5 - -36.5 -36 - -36 -35.5 - -35.5 -35 - -35 -34.5 - -34.5 -34 - -34 -33.5 - -33.5 -33 - -33 -32.5 - -32.5 -32 - -32 -31.5 - -31.5 -31 - -31 -30.5 - -30.5 -30 - -30 -29.5 - -29.5 -29 - -29 -28.5 - -28.5 -28 - -28 -27.5 - -27.5 -27 - -27 -26.5 - -26.5 -26 - -26 -25.5 - -25.5 -25 - -25 -24.5 - -24.5 -24 - -24 -23.5 - -23.5 -23 - -23 -22.5 - -22.5 -22 - -22 -21.5 - -21.5 -21 - -21 -20.5 - -20.5 -20 - -20 -19.5 - -19.5 -19 - -19 -18.5 - -18.5 -18 - -18 -17.5 - -17.5 -17 - -17 -16.5 - -16.5 -16 - -16 -15.5 - -15.5 -15 - -15 -14.5 - -14.5 -14 - -14 -13.5 - -13.5 -13 - -13 -12.5 - -12.5 -12 - -12 -11.5 - -11.5 -11 - -11 -10.5 - -10.5 -10 - -10 -9.5 - -9.5 -9 - -9 -8.5 - -8.5 -8 - -8 -7.5 - -7.5 -7 - -7 -6.5 - -6.5 -6 - -6 -5.5 - -5.5 -5 - -5 -4.5 - -4.5 -4 - -4 -3.5 - -3.5 -3 - -3 -2.5 - -2.5 -2 - -2 -1.5 - -1.5 -1 - -1 -0.5 - -0.5 0 - 0 0.5 - 0.5 1 - 1 1.5 - 1.5 2 - 2 2.5 - 2.5 3 - 3 3.5 - 3.5 4 - 4 4.5 - 4.5 5 - 5 5.5 - 5.5 6 - 6 6.5 - 6.5 7 - 7 7.5 - 7.5 8 - 8 8.5 - 8.5 9 - 9 9.5 - 9.5 10 - 10 10.5 - 10.5 11 - 11 11.5 - 11.5 12 - 12 12.5 - 12.5 13 - 13 13.5 - 13.5 14 - 14 14.5 - 14.5 15 - 15 15.5 - 15.5 16 - 16 16.5 - 16.5 17 - 17 17.5 - 17.5 18 - 18 18.5 - 18.5 19 - 19 19.5 - 19.5 20 - 20 20.5 - 20.5 21 - 21 21.5 - 21.5 22 - 22 22.5 - 22.5 23 - 23 23.5 - 23.5 24 - 24 24.5 - 24.5 25 - 25 25.5 - 25.5 26 - 26 26.5 - 26.5 27 - 27 27.5 - 27.5 28 - 28 28.5 - 28.5 29 - 29 29.5 - 29.5 30 - 30 30.5 - 30.5 31 - 31 31.5 - 31.5 32 - 32 32.5 - 32.5 33 - 33 33.5 - 33.5 34 - 34 34.5 - 34.5 35 - 35 35.5 - 35.5 36 - 36 36.5 - 36.5 37 - 37 37.5 - 37.5 38 - 38 38.5 - 38.5 39 - 39 39.5 - 39.5 40 - 40 40.5 - 40.5 41 - 41 41.5 - 41.5 42 - 42 42.5 - 42.5 43 - 43 43.5 - 43.5 44 - 44 44.5 - 44.5 45 - 45 45.5 - 45.5 46 - 46 46.5 - 46.5 47 - 47 47.5 - 47.5 48 - 48 48.5 - 48.5 49 - 49 49.5 - 49.5 50 - 50 50.5 - 50.5 51 - 51 51.5 - 51.5 52 - 52 52.5 - 52.5 53 - 53 53.5 - 53.5 54 - 54 54.5 - 54.5 55 - 55 55.5 - 55.5 56 - 56 56.5 - 56.5 57 - 57 57.5 - 57.5 58 - 58 58.5 - 58.5 59 - 59 59.5 - 59.5 60 - 60 60.5 - 60.5 61 - 61 61.5 - 61.5 62 - 62 62.5 - 62.5 63 - 63 63.5 - 63.5 64 - 64 64.5 - 64.5 65 - 65 65.5 - 65.5 66 - 66 66.5 - 66.5 67 - 67 67.5 - 67.5 68 - 68 68.5 - 68.5 69 - 69 69.5 - 69.5 70 - 70 70.5 - 70.5 71 - 71 71.5 - 71.5 72 - 72 72.5 - 72.5 73 - 73 73.5 - 73.5 74 - 74 74.5 - 74.5 75 - 75 75.5 - 75.5 76 - 76 76.5 - 76.5 77 - 77 77.5 - 77.5 78 - 78 78.5 - 78.5 79 - 79 79.5 - 79.5 80 - 80 80.5 - 80.5 81 - 81 81.5 - 81.5 82 - 82 82.5 - 82.5 83 - 83 83.5 - 83.5 84 - 84 84.5 - 84.5 85 - 85 85.5 - 85.5 86 - 86 86.5 - 86.5 87 - 87 87.5 - 87.5 88 - 88 88.5 - 88.5 89 - 89 89.5 - 89.5 90