This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
working_groups:ces:postprocess [2023/02/07 10:17] ctena [20230207] |
working_groups:ces:postprocess [2023/05/02 08:09] (current) ctena |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== AC Post-Process Working Group Meetings ====== | ====== AC Post-Process Working Group Meetings ====== | ||
+ | |||
+ | ===== 20230502 ===== | ||
+ | Attendants: | ||
+ | |||
+ | **NES**: | ||
+ | * Alba requested **Pypi** to remove the current NES package. Stattus? | ||
+ | * **Unit changing** (Dene' | ||
+ | * Pending to provide examples to Dene. [[https:// | ||
+ | * **Rotated-Nested** create. Pending to merge [[https:// | ||
+ | * **MONARCH** output type | ||
+ | * Pending: **CMAQ** and **WRF-Chem** [[https:// | ||
+ | * **Tutorial** date selected: [[https:// | ||
+ | * 23/28 people | ||
+ | * **TODO**: Check & refactor Jupyther Notebooks [[https:// | ||
+ | * **Pending**: | ||
+ | * **Unmapped** horizontal interpolation regions [[https:// | ||
+ | |||
+ | **SNES**: | ||
+ | * Added **daily** input type form ESARCHIVE | ||
+ | * Added **conservative** horizontal interpolation | ||
+ | * TODO: agree and implement flux to mass (and mass to flux) for conservative remapping. | ||
+ | * We have suggested some options and the chosen one from Oriol is to add a new column in the CSV indicating if you want to apply that transformation. [[https:// | ||
+ | |||
+ | **auto-SNES**: | ||
+ | * Oriol is **testing the conservative** remapping with auto-SNES & AS/4. | ||
+ | * Dani Beltran suggest to remove the current AS/4 to point to the new tag, but I rejected the idea until auto-SNES is adapted to the new tag changes. [[https:// | ||
+ | * **Pending**: | ||
+ | * Testing suite [[https:// | ||
+ | |||
+ | **Other stuff** | ||
+ | * **Output-checker**: | ||
+ | * Slow multi-day statistics (Miriam) | ||
+ | * Maria' | ||
+ | * Providentia Interpolation [[https:// | ||
+ | |||
+ | |||
+ | ===== 20230404 ===== | ||
+ | Attendants: | ||
+ | |||
+ | **NES** | ||
+ | * Alba suggests opening the NES library with PyPi: | ||
+ | * Readthedocs documentation | ||
+ | * Change the name to one not picked in the PyPi repository | ||
+ | * **PIOLES**: **P**arallel **I**nput **O**utput **L**ibrary for **E**arth **S**cience | ||
+ | * PIOLib | ||
+ | * PIOLIbES | ||
+ | * **APIO**: **A**utomatic **P**arallel **I**nput **O**utput | ||
+ | * **PIOESlib**: | ||
+ | * **MIOPES**: **M**ulti **IO** **P**arallel for **E**arth **S**cience | ||
+ | * Unit changing (Dene' | ||
+ | * Alba is going to lead that task. | ||
+ | * | ||
+ | |||
+ | |||
+ | **SNES** | ||
+ | * No news | ||
+ | |||
+ | **auto-SNES** | ||
+ | * No news | ||
+ | |||
+ | **Other stuff** | ||
+ | * Slow multi-day statistics (Miriam) | ||
+ | * Maria' | ||
+ | * Providentia Interpolation | ||
+ | |||
+ | |||
+ | ===== 20230321 ===== | ||
+ | Attendants: | ||
+ | |||
+ | **NES** | ||
+ | * DONE: | ||
+ | * Bugfix on cell_methods serial write | ||
+ | * Write 2D string data (timezones) | ||
+ | * Sum Nes objects (sum variables) | ||
+ | * Write by time step | ||
+ | * Concatenation improved times | ||
+ | * Working on: | ||
+ | * Spin-up bug | ||
+ | * TODO: | ||
+ | * Nested-Rotated domain (needed for HERMES) | ||
+ | * Spatial Join: Improve time & memory for large datasets (needed for HERMES) | ||
+ | * Training requested. | ||
+ | |||
+ | **SNES** | ||
+ | * No news | ||
+ | |||
+ | **Auto-SNES** | ||
+ | * Testing suite experiments created. | ||
+ | * Next steps: | ||
+ | * Fix the benchmark experiments (waiting for auto-MONARCH validation) | ||
+ | |||
+ | **Auto-MONARCH** | ||
+ | * Testing Suite. Output checker fails for NES_master | ||
+ | |||
+ | **Other stuff** | ||
+ | * | ||
+ | |||
+ | ===== 20230307 ===== | ||
+ | Attendants: | ||
+ | |||
+ | **NES** | ||
+ | * v1.1.0 release done: | ||
+ | * Improve Lat-Lon to Cartesian coordinates method (used in Providentia). | ||
+ | * Horizontal Interpolation: | ||
+ | * Shapefile: | ||
+ | * Function **to_shapefile()** to create shapefiles from a NES object without losing data from the original grid and being able to select the time and level. | ||
+ | * Function **from_shapefile()** to create a new grid with data from a shapefile after doing a spatial join. | ||
+ | * Function **create_shapefile()** can now be used in parallel. | ||
+ | * Function **calculate_grid_area()** to calculate the area of each cell in a grid. | ||
+ | * Function **calculate_geometry_area()** to calculate the area of each cell given a set of geometries. | ||
+ | * Function **get_spatial_bounds_mesh_format()** to get the lon-lat boundaries in a mesh format (used in pcolormesh). | ||
+ | * Bugs fixing: | ||
+ | * Correct the dimensions of the resulting points datasets from any interpolation. | ||
+ | * Amend the interpolation method to take into account the cases in which the distance among points equals zero. | ||
+ | * Correct the way we retrieve the level positive value. | ||
+ | * Correct how to calculate the spatial bounds of LCC and Mercator grids: the dimensions were flipped. | ||
+ | * Correct how to calculate the spatial bounds for all grids: use read axis limits instead of write axis limits. | ||
+ | * Calculate centroids from coordinates in creating shapefiles, instead of using the geopandas function ' | ||
+ | * Enable selection of variables on the creation of shapefiles. | ||
+ | * Correct read and write parallel limits. | ||
+ | * Correct data type in the parallelization of points datasets. | ||
+ | * Correct errors that appear when trying to select coordinates and write the file. | ||
+ | * Working on: | ||
+ | * Pending to merge: | ||
+ | * Bug on cell_methods serial write | ||
+ | * Next steps: | ||
+ | * Sum Nes objects (sum variables) | ||
+ | * Write 2D string data (timezones) | ||
+ | * Nested-Rotated domain | ||
+ | * Training requested. | ||
+ | |||
+ | **SNES** | ||
+ | * v1.1.0 release done: | ||
+ | * Correct way to use level.positive NetCDF metadata. | ||
+ | * Added sum_levels new options in the CSV. | ||
+ | * Added the option to choose if you want to enable (or not) the sum_levels option by variable with the ' | ||
+ | * Added the option to choose if you want to write (or not) the sum_levels option by variable with the ' | ||
+ | * Added NES conservative horizontal interpolation. | ||
+ | * NES update to v1.1.0 (see NESv1.1.0 release notes) | ||
+ | * Next steps: | ||
+ | * run an auto-MONARCH testing suite | ||
+ | |||
+ | **Auto-SNES** | ||
+ | * Next steps: | ||
+ | * Testing suite: | ||
+ | * Create a testing suite with operational runs (at least) | ||
+ | |||
+ | **Auto-MONARCH** | ||
+ | * Everything OK? | ||
+ | |||
+ | **Other stuff** | ||
+ | * | ||
+ | |||
===== 20230207 ===== | ===== 20230207 ===== |