Order of dimensions changed for heatsum
2 files generated with the same config file but for different dates have a different order of dimension, preventing them from properly concatenating in the monitor job.
Discovered in t0e7:
pbretonn@bscearth319:/esarchive/exp/ecearth/t0e7/diags/DCPP/EC-Earth-Consortium/EC-Earth3/dcppA-hindcast/r9i1p1f1$ ncdump -h ./Omon/heatcsum0-700m/gn/v20240116/heatcsum0-700m_Omon_EC-Earth3_dcppA-hindcast_s2001-r9i1p1f1_gn_200101-200112.nc
netcdf heatcsum0-700m_Omon_EC-Earth3_dcppA-hindcast_s2001-r9i1p1f1_gn_200101-200112 {
dimensions:
region = 5 ;
time = 12 ;
region_length = 15 ;
variables:
double time(time) ;
time:axis = "T" ;
time:units = "days since 1850-01-01 00:00:00" ;
time:standard_name = "time" ;
time:long_name = "time" ;
time:calendar = "proleptic_gregorian" ;
double heatcsum0-700m(**region, time**) ;
heatcsum0-700m:long_name = "Total Ocean heat content" ;
heatcsum0-700m:units = "J" ;
heatcsum0-700m:coordinates = "region time" ;
heatcsum0-700m:standard_name = "total_ocean_heat_content" ;
heatcsum0-700m:_FillValue = 1.e+20 ;
heatcsum0-700m:missingValue = 1.e+20 ;
char region(region, region_length) ;
region:standard_name = "region" ;
// global attributes:
:invalid_standard_name = "total_ocean_heat_content" ;
:missingValue = 1.e+20 ;
:modeling_realm = "ocean" ;
:table_id = "Table Omon (December 2013)" ;
:Conventions = "CF-1.7" ;
:NCO = "netCDF Operators version 4.9.9 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)" ;
:history = "UTC 2024-01-18T07:30:49.514896: Diagnostic Heat content layer Startdate: 20010101 Member: 9 Chunk: 1 Box: 0-700m calculated with EarthDiagnostics version 3.5.9;UTC 2024-01-18T07:30:49.512999: Using Basins masks file version with grid Ec3.2_O1L75. Original file can be found in /esarchive/autosubmit/conf_files.;Thu Jan 18 08:30:49 2024: /gpfs/projects/bsc32/software/suselinux/11/software/earthdiagnostics/3.5.9/bin/ncatted -O -a _FillValue,heatcsum0-700m,o,d,1.e20 -a missingValue,heatcsum0-700m,o,d,1.e20 --nco_dbg_lvl=3 --output=/scratch/tmp/2730916/diags/t0e7/templuj2p2cc.nc /scratch/tmp/2730916/diags/t0e7/templuj2p2cc.nc\nThu Jan 18 08:30:49 2024: /gpfs/projects/bsc32/software/suselinux/11/software/earthdiagnostics/3.5.9/bin/ncatted -O -a _FillValue,heatcsum0-700m,o,d,1.e20 -a missingValue,heatcsum0-700m,o,d,1.e20 --nco_dbg_lvl=3 --output=/scratch/tmp/2730916/diags/t0e7/templuj2p2cc.nc /scratch/tmp/2730916/diags/t0e7/templuj2p2cc.nc" ;
:_NCProperties = "version=2,netcdf=4.7.4,hdf5=1.10.6" ;
}
(base) pbretonn@bscearth319:/esarchive/exp/ecearth/t0e7/diags/DCPP/EC-Earth-Consortium/EC-Earth3/dcppA-hindcast/r9i1p1f1$ ncdump -h ./Omon/heatcsum0-700m/gn/v20240116/heatcsum0-700m_Omon_EC-Earth3_dcppA-hindcast_s2001-r9i1p1f1_gn_200201-200212.nc
netcdf heatcsum0-700m_Omon_EC-Earth3_dcppA-hindcast_s2001-r9i1p1f1_gn_200201-200212 {
dimensions:
time = 12 ;
region = 5 ;
region_length = 50 ;
variables:
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-01-01 00:00:00" ;
time:calendar = "proleptic_gregorian" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
char region(region, region_length) ;
double heatcsum0-700m(**time, region**) ;
heatcsum0-700m:units = "J" ;
heatcsum0-700m:coordinates = "region time" ;
heatcsum0-700m:standard_name = "total_ocean_heat_content" ;
heatcsum0-700m:long_name = "Total Ocean heat content" ;
heatcsum0-700m:_FillValue = 1.e+20 ;
heatcsum0-700m:missingValue = 1.e+20 ;
// global attributes:
:modeling_realm = "ocean" ;
:table_id = "Table Omon (December 2013)" ;
:NCO = "netCDF Operators version 4.9.9 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)" ;
:history = "UTC 2024-01-18T07:16:40.625440: Diagnostic Heat content layer Startdate: 20010101 Member: 9 Chunk: 2 Box: 0-700m calculated with EarthDiagnostics version 3.5.9;UTC 2024-01-18T07:16:40.623510: Using Basins masks file version with grid Ec3.2_O1L75. Original file can be found in /esarchive/autosubmit/conf_files.;Thu Jan 18 08:16:40 2024: /gpfs/projects/bsc32/software/suselinux/11/software/earthdiagnostics/3.5.9/bin/ncatted -O -a _FillValue,heatcsum0-700m,o,d,1.e20 -a missingValue,heatcsum0-700m,o,d,1.e20 --nco_dbg_lvl=3 --output=/scratch/tmp/2730895/diags/t0e7/tempw6g88lus.nc /scratch/tmp/2730895/diags/t0e7/tempw6g88lus.nc" ;
:_NCProperties = "version=2,netcdf=4.7.4,hdf5=1.10.6" ;
Rerunning the job fixes it (to be confirmed by @eferre1 ) and the job is anyway being replaced by ESMValTool by @sghosh but commenting it here for proper documentation