At the Climate Services team, we would like to compute the Continuous Ranked Probability Score (CRPS) and its associated Skill Score (CRPSS) to evaluate the quality of probabilistic forecasts. We have been using SpecsVerification::EnsCrps() so far, but we would like to include this metric into s2dv for convenience. These metrics are very similar to RPS and RPSS, but CRPS and CRPSS evaluate the full probability distribution instead of (discrete) multi-categorical forecasts.
Basically, we will work with named dimensions instead of dimension indices (as for the rest of the metrics/functions in s2dv), and we will compute the "fair" version correctly (in SpecsVerification, the "fair" correction is applied to the forecast, but not to the reference forecast). Also, we will have the possibility of computing the Skill Score with respect to the climatological forecast (this feature is not included in SpecsVerification).
I have included two new functions in the branch develop-CRPSS of s2dv: CRPS (https://earth.bsc.es/gitlab/es/s2dv/-/blob/develop-CRPSS/R/CRPS.R) and CRPSS (https://earth.bsc.es/gitlab/es/s2dv/-/blob/develop-CRPSS/R/CRPSS.R). The atomic functions call _SpecsVerification_enscrps_cpp, implemented in C++, to compute the CRPS. The statistical significance of the CRPSS is obtained with the Random Walk test.
The functions already have documentation (mainly copied from RPS and RPSS), basic checks, and they are in an s2dv-like format.
@vagudets agreed with checking and testing the function (thanks a lot!!), but I'm tagging here people who can also be interested in doing some tests and/or providing feedback. @nmilders @allabres @ngonzal2 @jcos @nperez @jramon @lpalma @cchou @bsolaraj
Thank you!
Carlos