Skip to content
GitLab

Unit test for sub-seasonal crossval calibration

Merged Nuria Pérez-Zanón requested to merge dev_subseasonal-cal into master

Hi @vagudets

@vtorralba is trying to use the operational code for sub-seasonal forecast (https://earth.bsc.es/gitlab/es/sunset/-/blob/master/operationals/bigpredidata/recipe_bigpredidata_oper_subseasonal.yml) and she is finding some problems.

I have solved two of them and created a unit test based on the existing sub-seasonal recipe (https://earth.bsc.es/gitlab/es/sunset/-/blob/master/tests/recipes/recipe-subseasonal_weekly.yml):

  1. When merging 'sday' and 'syear' dimensions, na.rm should be FALSE to keep always the same length.
  2. The crps_clim and crpss had some problems for the sub-seasonal case

The existing recipe for the test states the saving probabilities "percentile_only", but in the BigPrediData case, it also saves probabilities, and this is failing. @vtorralba may open an issue about that, but you might be willing to test it as well. In any case, the current unit test worked in the WS, so it can be merged if you want.

For the future, we may need to consider the difference between having NAs because they are part of the dataset or because a file is missing. Some metrics are returned now, like mean bias and correlation, which I am not totally sure if they are correct. A warning may be useful in this case.

I hope this helps,

Núria