Commits · 0fa8b38b91c879dd8a475d836407ee595abb42a4 · Marvin Axness / Post Processing Netcdf Files · GitLab

14 Mar, 2025 13 commits
- Adding flush to print · 0fa8b38b
  Marvin Axness authored Mar 14, 2025
  
  0fa8b38b
- Parameter number data_chcker combine netcdf · 1647dd0c
  Marvin Axness authored Mar 14, 2025
  
  1647dd0c
- Adding debug print · e97a9578
  Marvin Axness authored Mar 14, 2025
  
  e97a9578
- Added code to reproduce bottleneck · 293df36b
  Marvin Axness authored Mar 14, 2025
  
  293df36b
- Adding touch to create empty file right before writing, which allows for multiple job executions · 97e26152
  Marvin Axness authored Mar 14, 2025
  
  97e26152
- Created full exp analysis slurm + took out chunking · 1adec689
  Marvin Axness authored Mar 14, 2025
  
  1adec689
- Bug for chunk_start_date value reading from Daily fieds · 1b382d19
  Marvin Axness authored Mar 14, 2025
  
  1b382d19
- PostProcess only hourly and monthly data · 69ac8f64
  Marvin Axness authored Mar 14, 2025
  
  69ac8f64
- Chunking for writing file · f2d7f0a4
  Marvin Axness authored Mar 14, 2025
  
  f2d7f0a4
- working on MN5, little parallelization · 6a591563
  Marvin Axness authored Mar 14, 2025
  
  6a591563
- Adding code to loop through all successful rundirs · 313925a7
  Marvin Axness authored Mar 14, 2025
  
  313925a7
- Eliminating level parallelization for now · b9880157
  Marvin Axness authored Mar 14, 2025
  
  b9880157
- Dask parallelization during writing · 442c5004
  Marvin Axness authored Mar 14, 2025
  
  442c5004
13 Mar, 2025 13 commits
- Looking into chunking when writing nc file · 38ad8264
  Marvin Axness authored Mar 13, 2025
  
  38ad8264
- Added parallelization to dates and levels with n_proc var · d77d4c3d
  Marvin Axness authored Mar 13, 2025
  
  d77d4c3d
- Added level parallelization · af0961c3
  Marvin Axness authored Mar 13, 2025
  
  af0961c3
- Force dask to use single-threaded execution ( we loose parallelization) · f11ca070
  Marvin Axness authored Mar 13, 2025
  
  f11ca070
- Adding chunking · b328fa14
  Marvin Axness authored Mar 13, 2025
  
  b328fa14
- Reverting engine used from h5netcdf to netcdf4 · 7136476e
  Marvin Axness authored Mar 13, 2025
  
  7136476e
- Error handling in processing level fields · 2f658340
  Marvin Axness authored Mar 13, 2025
  
  2f658340
- Small argument fixes and new submission script slurm · d499197d
  Marvin Axness authored Mar 13, 2025
  
  d499197d
- Add path to cmor table · 4e70687b
  Marvin Axness authored Mar 13, 2025
  
  4e70687b
- Adding print debug line · 7478fe76
  Marvin Axness authored Mar 13, 2025
  
  7478fe76
- Apply dask delayed for all calls in combine_netcdf · 27719b9f
  Marvin Axness authored Mar 13, 2025
  
  27719b9f
- Parallelizing inner level loop with dask · d3f321fa
  Marvin Axness authored Mar 13, 2025
  
  d3f321fa
- Added NEMO layer conversion from level to depth in m · 8a1ed721
  Marvin Axness authored Mar 13, 2025
  
  8a1ed721
12 Mar, 2025 4 commits
- Code improvements · 77c2db11
  Marvin Axness authored Mar 12, 2025
  
  77c2db11
- Important changes to get design working · fbcfdb3b
  Marvin Axness authored Mar 12, 2025
  
  fbcfdb3b
- New design almost ready, missing parallelization · d42a9e5b
  Marvin Axness authored Mar 12, 2025
  
  d42a9e5b
- Lots of code developments for different postprocessing design · 3cd03712
  Marvin Axness authored Mar 12, 2025
  
  3cd03712
11 Mar, 2025 1 commit
- New code design for postproc eerie · e0b16da3
  Marvin Axness authored Mar 11, 2025
  
  e0b16da3
07 Mar, 2025 9 commits
- Separating parts of code · 1537eb41
  Marvin Axness authored Mar 07, 2025
  
  1537eb41
- Changing design · fa6ae447
  Marvin Axness authored Mar 07, 2025
  
  fa6ae447
- Improving code · 6536b564
  Marvin Axness authored Mar 07, 2025
  
  6536b564
- Fix · dfde577d
  Marvin Axness authored Mar 07, 2025
  
  dfde577d
- Some performance improvements from glob search pattern · 4e5b6655
  Marvin Axness authored Mar 07, 2025
  
  4e5b6655
- Some slight performance improvements · bc2c289f
  Marvin Axness authored Mar 07, 2025
  
  bc2c289f
- Indent error, also need to improve performance of file search · b79fcdb5
  Marvin Axness authored Mar 07, 2025
  
  b79fcdb5
- Analyze run with generate portfolio and datachecker ready? · d2c6a88c
  Marvin Axness authored Mar 07, 2025
  
  d2c6a88c
- Adding pipeline for data-portfolio generation, and file_checks · ce338a24
  Marvin Axness authored Mar 07, 2025
  
  ce338a24